Variable-interval schedules of timeout from avoidance

Rats were trained on concurrent schedules in which pressing one lever postponed shock and pressing the other occasionally produced a 2-min timeout during which the shock-postponement schedule was suspended and its correlated stimuli were removed. Throughout, the shock-postponement schedule maintained proficient levels of avoidance. Nevertheless, in Experiment 1 responding on the timeout lever was established rapidly, was maintained at stable levels on variable-interval schedules, was extinguished by withholding timeout, was reestablished when timeout was reintroduced, and was brought under discriminative control with a multiple variable-interval extinction schedule of timeout. These results are in contrast with Verhave's (1962) conclusion that timeout is an ineffective reinforcer when presented to rats on intermittent schedules. In Experiment 2 the consequence of responding on the timeout lever was altered so that the shock-postponement schedule remained in effect even though the stimulus conditions associated with timeout were produced for 2 min. Responding extinguished, indicating that suspension of the shock-postponement schedule, not stimulus change, was the source of reinforcement. By establishing the reinforcing efficacy of timeout with standard variable-interval schedules, these experiments illustrate a procedure for studying negative reinforcement in the same way as positive reinforcement.     

Concurrent choice: Effects of overall reinforcer rate and the temporal distribution of reinforcers

Six pigeons responded on a series of concurrent exponential variable-interval schedules, offering a within-subject comparison with previously published data from concurrent arithmetic variable-interval schedules. Both relative and overall reinforcer rates were varied between conditions. The generalized matching law described the data well, with undermatching much more frequent than strict matching. Time-allocation sensitivity consistently exceeded response-allocation sensitivity for both schedule types, and exponential-schedule sensitivity exceeded arithmetic-schedule sensitivity for both measures of choice. A further set of conditions using variable-interval schedules whose shortest interval was correlated with the mean interval, like arithmetic schedules, but that provided a constant conditional probability of reinforcement, like exponential schedules, produced sensitivities between those produced by conventional arithmetic and exponential schedules. Unlike previous arithmetic-schedule results, exponential sensitivity changed nonmonotonically with changes in overall reinforcer rate. The results clarify our knowledge of the effects of arithmetic and exponential schedules but confuse our understanding of the effects of overall reinforcer rate on concurrent choice.     

Human observing: maintained by negative informative stimuli only if correlated with improvement in response efficiency.

Two experiments investigated the effect of observing responses that enabled college students to emit more efficient distributions of reinforced responses. In Experiment 1, the gains of response efficiency enabled by observing were minimized through use of identical low-effort response requirements in two alternating variable-interval schedules. These comprised a mixed schedule of reinforcement; they differed in the number of money-backed points per reinforcer. In each of three choices between two stimuli that varied in their correlation with the variable-interval schedules, the results showed that subjects preferred stimuli that were correlated with the larger average amount of reinforcement. This is consistent with a conditioned-reinforcement hypothesis. Negative informative stimuli--that is, stimuli correlated with the smaller of two rewards--did not maintain as much observing as stimuli that were uncorrelated with amount of reward. In Experiment 2, savings in effort made possible by producing S- were varied within subjects by alternately removing and reinstating the response-reinforcement contingency in a mixed variable-interval/extinction schedule of reinforcement. Preference for an uncorrelated stimulus compared to a negative informative stimulus (S-) decreased for each of six subjects, and usually reversed when observing permitted a more efficient temporal distribution of the responses required for reinforcement; in this case, the responses were pulls on a relatively high-effort plunger. When observing the S- could not improve response efficiency, subjects again chose the control stimulus. All of these results were inconsistent with the uncertainty-reduction hypothesis.     

Changeover behavior and preference in concurrent schedules

Pigeons were trained on a multiple schedule of reinforcement in which separate concurrent schedules occurred in each of two components. Key pecking was reinforced with milo. During one component, a variable-interval 40-s schedule was concurrent with a variable-interval 20-s schedule; during the other component, a variable-interval 40-s schedule was concurrent with a variable-interval 80-s schedule. During probe tests, the stimuli correlated with the two variable-interval 40-s schedules were presented simultaneously to assess preference, measured by the relative response rates to the two stimuli. In Experiment 1, the concurrently available variable-interval 20-s schedule operated normally; that is, reinforcer availability was not signaled. Following this baseline training, relative response rate during the probes favored the variable-interval 40-s alternative that had been paired with the lower valued schedule (i.e., with the variable-interval 80-s schedule). In Experiment 2, a signal for reinforcer availability was added to the high-value alternative (i.e., to the variable-interval 20-s schedule), thus reducing the rate of key pecking maintained by that schedule but leaving the reinforcement rate unchanged. Following that baseline training, relative response rates during probes favored the variable-interval 40-s alternative that had been paired with the higher valued schedule. The reversal in the pattern of preference implies that the pattern of changeover behavior established during training, and not reinforcement rate, determined the preference patterns obtained on the probe tests.     

Concurrent second-order schedules of reinforcement

Responses on one key (the main key) of a two-key chamber produced food according to a second-order variable-interval schedule with fixed-interval schedule components. A response on a second key (the changeover key) alternated colors on the main key and provided a second independent second-order variable-interval schedule with fixed-interval components. The fixed-interval component on one variable-interval schedule was held constant at 8 sec, while the fixed interval on the other variable-interval schedule was varied from 0 to 32 sec. Under some conditions, a brief stimulus terminated each fixed interval and generated fixed-interval patterns; in other conditions, the brief stimulus was omitted. Relative response rate and relative time deviated substantially from scheduled relative reinforcement rate and, to a lesser extent, from obtained relative reinforcement rate under both brief-stimulus and no-stimulus conditions. Matching was observed with equal components on both schedules; with unequal components, increasingly greater proportions of time and responses than the matching relation would predict were spent on the variable-interval schedule containing the shorter component. Preference for the shorter fixed interval was typically more extreme under brief-stimulus than under no-stimulus schedules. The results limit the extension of the matching relation typically observed under simple concurrent variable-interval schedules to concurrent second-order variable-interval schedules.     

Effects on concurrent performances of a stimulus correlated with reinforcer availability

A multiple schedule was arranged in which each component consisted of two, concurrent variable-interval schedules of reinforcement. A changeover-key procedure was used, and the components of the multiple schedule were distinguished (initially) by the color of the changeover key. During one component of the multiple schedule, the availability of a reinforcer arranged by one of the variable-interval schedules was marked by an exteroceptive stimulus, provided that that variable-interval schedule was not at the time assigned to the main key. During the other component of the multiple schedule, no reinforcer-correlated stimuli were ever presented. During the latter component of the multiple schedule, the distribution of responses and time for the concurrent variable-interval schedules suggested control by the distribution of reinforcements. During the former component, most main-key responses were emitted on the key in the presence of which reinforcer-correlated stimuli were presented. Changeover rate in the presence of that key color was depressed. The discriminative control over the changeover was easily established and was reversible.     

Sensory superstition on multiple interval schedules.

Pigeons were exposed to multiple schedules in which an irregular repeating sequence of five stimulus components was correlated with the same reinforcement schedule throughout. Stable, idiosyncratic, response-rate differences developed across components. Components were rank-ordered by response rate; an approximately linear relation was found between rank order and the deviation of mean response rate from the overall mean rate. Nonzero slopes of this line were found for multiple fixed-interval and variable-time schedules and for multiple variable-interval schedules both when number of reinforcements was the same in all components and when it varied. The steepest function slopes were found in the variable schedules with relatively long interfood intervals and relatively short component durations. When just one stimulus was correlated with all components of a multiple variable-interval schedule, the slope of the line was close to zero. The results suggest that food-rate differences may be induced initially by different reactions to the stimuli and subsequently maintained by food.     

Variable-ratio versus variable-interval schedules: response rate, resistance to change, and preference

Two experiments asked whether resistance to change depended on variable-ratio as opposed to variable-interval contingencies of reinforcement and the different response rates they establish. In Experiment 1, pigeons were trained on multiple random-ratio random-interval schedules with equated reinforcer rates. Baseline response rates were disrupted by intercomponent food, extinction, and prefeeding. Resistance to change relative to baseline was greater in the interval component, and the difference was correlated with the extent to which baseline response rates were higher in the ratio component. In Experiment 2, pigeons were trained on multiple variable-ratio variable-interval schedules in one half of each session and on concurrent chains in the other half in which the terminal links corresponded to the multiple-schedule components. The schedules were varied over six conditions, including two with equated reinforcer rates. In concurrent chains, preference strongly overmatched the ratio of obtained reinforcer rates. In multiple schedules, relative resistance to response-independent food during intercomponent intervals, extinction, and intercomponent food plus extinction depended on the ratio of obtained reinforcer rates but was less sensitive than was preference. When reinforcer rates were similar, both preference and relative resistance were greater for the variable-interval schedule, and the differences were correlated with the extent to which baseline response rates were higher on the variable-ratio schedule, confirming the results of Experiment 1. These results demonstrate that resistance to change and preference depend in part on response rate as well as obtained reinforcer rate, and challenge the independence of resistance to change and preference with respect to response rate proposed by behavioral momentum theory.     

Conditioned reinforcement and discrimination in second-order schedules

Pigeons were exposed to multiple second-order schedules in which responding on the “main key” was reinforced according to either a variable-interval or fixed-interval schedule by production of a brief stimulus on the “brief-stimulus key”. A response was required to the brief stimulus during its fourth (final) presentation to produce food; responses to the earlier brief stimuli indicated the extent to which the final brief stimulus was discriminated from preceding ones. Main-key response rates were higher in early components of paired brief-stimulus schedules, in which each brief stimulus was the same as that paired with reinforcement, than in comparable unpaired brief-stimulus or tandem schedules. Poor discrimination occurred between paired brief stimuli (Experiment I). When chain stimuli on the main key induced a discrimination between the first two and second two brief stimuli, the response-rate enhancement in the paired brief-stimulus schedule persisted (Experiment II). Rate enhancement diminished when the initial link of the chain included the first three components (Experiment IV). Eliminating the contingency between responding and brief-stimulus production also diminished rate enhancement (Experiment III). The results show that the discriminative and conditioned reinforcing effects of food-paired brief stimuli may be selectively manipulated and suggest that the reinforcing effects are modulated by other reinforcers in the situation.     

Choice behavior and the accessibility of the reinforcer

In Experiment 1, matching of relative response rates to relative rates of reinforcement was obtained in concurrent variable-interval schedules when the absolute values of the two concurrent variable-interval schedules varied from 6 sec and 12 sec to 600 sec and 1200 sec. Increases in the duration of the changeover delay, however, produced decreases in the relative response rates and, consequently, some deviation from matching. In Experiment 2, matching of relative response rates to the relative duration of the reinforcer failed to occur when the equal variable-interval schedules arranging access to the two different reinforcer durations (1.5 and 6 sec) were varied in size from concurrent variable-interval 10-sec schedules to concurrent variable-interval 600-sec schedules.     

Choice between response units: The rate constancy model

In a conjoint schedule, reinforcement is available simultaneously on two or more schedules for the same response. The present experiments provided food for key pecking on both a random-interval and a differential-reinforcement-of-low-rate (DRL) schedule. Experiment 1 involved ordinary DRL schedules; Experiment 2 added an external stimulus to indicate when the required interresponse time had elapsed. In both experiments, the potential reinforcer frequency from each component was varied by means of a second-order fixed-ratio schedule, and the DRL time parameter was changed as well. Response rates were described by a model stating that time allocation to each component matches the relative frequency of reinforcement for that component. When spending time in a given component, the subject is assumed to respond at the rate characteristic of baseline performance. This model appeared preferable to the absolute-rate version of the matching law. The model was shown to be applicable to multiple-response concurrent schedules as well as to conjoint schedules, and it described some of the necessary conditions for response matching, undermatching, and bias. In addition, the pigeons did not optimize reinforcer frequency.     

Sensitivity to relative reinforcer rate in concurrent schedules: independence from relative and absolute reinforcer duration

Twelve pigeons responded on two keys under concurrent variable-interval (VI) schedules. Over several series of conditions, relative and absolute magnitudes of reinforcement were varied. Within each series, relative rate of reinforcement was varied and sensitivity of behavior ratios to reinforcer-rate ratios was assessed. When responding at both alternatives was maintained by equal-sized small reinforcers, sensitivity to variation in reinforcer-rate ratios was the same as when large reinforcers were used. This result was observed when the overall rate of reinforcement was constant over conditions, and also in another series of concurrent schedules in which one schedule was kept constant at VI ached 120 s. Similarly, reinforcer magnitude did not affect the rate at which response allocation approached asymptote within a condition. When reinforcer magnitudes differred between the two responses and reinforcer-rate ratios were varied, sensitivity of behavior allocation was unaffected although response bias favored the schedule that arranged the larger reinforcers. Analysis of absolute response rates ratio sensitivity to reinforcement occurrred on the two keys showed that this invariance of response despite changes in reinforcement interaction that were observed in absolute response rates on the constant VI 120-s schedule. Response rate on the constant VI 120-s schedule was inversely related to reinforcer rate on the varied key and the strength of this relation depended on the relative magnitude of reinforcers arranged on varied key. Independence of sensitivity to reinforcer-rate ratios from relative and absolute reinforcer magnitude is consistent with the relativity and independence assumtions of the matching law.     

Stimulus Effects On Behavior Allocation In Three-alternative Choice

Six pigeons were trained on three-alternative concurrent variable-interval schedules that were available through a switching response and were signaled by colored stimuli. The discriminative stimuli for two of the schedules were always 560 nm and 630 nm, but the stimulus signaling the third alternative was varied across conditions over seven levels between these colors. For each third-alternative stimulus condition, the relative frequency of reinforcers was varied over three conditions with 4:1 and 16:1 reinforcer ratios between each pair of alternatives. The distribution of responses between the alternatives was dependent jointly on the third-alternative reinforcer rate and on the disparity between the stimulus signaling the third alternative and those signaling the other alternatives. A generalized matching approach was unable to provide invariant measures of the discriminability between constant stimuli, but a contingency-discriminability approach provided excellent fits and sensible and invariant stimulus discriminability measures.     

Matching: its acquisition and generalization

Choice typically is studied by exposing organisms to concurrent variable-interval schedules in which not only responses controlled by stimuli on the key are acquired but also switching responses and likely other operants as well. In the present research, discriminated key-pecking responses in pigeons were first acquired using a multiple schedule that minimized the reinforcement of switching operants. Then, choice was assessed during concurrent-probe periods in which pairs of discriminative stimuli were presented concurrently. Upon initial exposure to concurrently presented stimuli, choice approximated exclusive preference for the alternative associated with the higher reinforcement frequency. Concurrent schedules were then implemented that gave increasingly greater opportunities for switching operants to be conditioned. As these operants were acquired, the relation of relative response frequency to relative reinforcement frequency converged toward a matching relation. An account of matching with concurrent schedules is proposed in which responding exclusively to the discriminative stimulus associated with the higher reinforcement frequency declines as the concurrent stimuli become more similar and other operants-notably switching-are acquired and generalize to stimuli from both alternatives. The concerted effect of these processes fosters an approximate matching relation in commonly used concurrent procedures.     

Stimulus Equivalence: Effects Of A Default-response Option On Emergence Of Untrained Stimulus Relations

Human subjects were exposed to a concurrent-chains schedule in which reinforcer amounts, delays, or both were varied in the terminal links, and consummatory responses were required to receive points that were later exchangeable for money. Two independent variable-interval 30-s schedules were in effect during the initial links, and delay periods were defined by fixed-time schedules. In Experiment 1, subjects were exposed to three different pairs of reinforcer amounts and delays, and sensitivity to reinforcer amount and delay was determined based on the generalized matching law. The relative responding (choice) of most subjects was more sensitive to reinforcer amount than to reinforcer delay. In Experiment 2, subjects chose between immediate smaller reinforcers and delayed larger reinforcers in five conditions with and without timeout periods that followed a shorter delay, in which reinforcer amounts and delays were combined to make different predictions based on local reinforcement density (i.e., points per delay) or overall reinforcement density (i.e., points per total time). In most conditions, subjects' choices were qualitatively in accord with the predictions from the overall reinforcement density calculated by the ratio of reinforcer amount and total time. Therefore, the overall reinforcement density appears to influence the preference of humans in the present self-control choice situation.     

The role of discriminative stimuli in concurrent performances

Key pecking in pigeons was examined under concurrent and parallel arrangements of two independent and simultaneously available variable-interval schedules. Pecks on the changeover key alternated the schedule of reinforcement for responses on the main key. Under concurrent schedules, discriminative stimuli were paired with the reinforcement schedule arranged in each component and changeover responses also alternated these stimuli. Under parallel schedules, changeover responses alternated the effective reinforcement schedule, but did not change the discriminative stimulus. On concurrent procedures, changeover response rate was inversely related to the difference in reinforcement rate between the two components, whereas on parallel schedules no consistent relationship was found. With both schedules, absolute response and reinforcement rates were positively related, although for a given set of reinforcement frequencies, rates were often higher on the concurrent schedules. On concurrent schedules, relative response rates and relative times were equal to relative reinforcement rates. On parallel schedules these ratios were positively related, but response and time ratios were much smaller than were obtained with comparable concurrent schedules. This inequality was most pronounced when absolute reinforcement frequencies were lowest.     

Effects of reinforcement rate and reinforcer magnitude on choice behavior of humans

Two experiments with human subjects investigated the effects of rate of reinforcement and reinforcer magnitude upon choice. In Experiment 1, each of five subjects responded on four concurrent variable-interval schedules. In contrast to previous studies using non-human organisms, relative response rate did not closely match relative rate of reinforcement. Discrepancies ranged from 0.03 to 0.43 (mean equal to 0.19). Similar discrepancies were found between relative amount of time spent responding on each schedule and the corresponding relative rates of reinforcement. In Experiment 2, in which reinforcer magnitude was varied for each of five subjects, similar discrepancies ranging from 0.05 to 0.50 (mean equal to 0.21), were found between relative response rate and relative proportion of reinforcers received. In both experiments, changeover rates were lower on the long-interval concurrent schedules than on the short-interval ones. The results suggest that simple application of previous generalizations regarding the effects of reinforcement rate and reinforcer magnitude on choice for variable-interval schedules does not accurately describe human behavior in a simple laboratory situation.     

Arousal, changeover responses, and preference in concurrent schedules

Pigeons were trained on multiple schedules that provided concurrent reinforcement in each of two components. In Experiment 1, one component consisted of a variable-interval (VI) 40-s schedule presented with a VI 20-s schedule, and the other a VI 40-s schedule presented with a VI 80-s schedule. After extended training, probe tests measured preference between the stimuli associated with the two 40-s schedules. Probe tests replicated the results of Belke (1992) that showed preference for the 40-s schedule that had been paired with the 80-s schedule. In a second condition, the overall reinforcer rate provided by the two components was equated by adding a signaled VI schedule to the component with the lower reinforcer rate. Probe results were unchanged. In Experiment 2, pigeons were trained on alternating concurrent VI 30-s VI 60-s schedules. One schedule provided 2-s access to food and the other provided 6-s access. The larger reinforcer magnitude produced higher response rates and was preferred on probe trials. Rate of changeover responding, however, did not differ as a function of reinforcer magnitude. The present results demonstrate that preference on probe trials is not a simple reflection of the pattern of changeover behavior established during training.     

Matching to Reinforcer Value: Human Concurrent Variable-Interval Performance

A male paedophile offender chose between pairs of sexual stimulus classes in a two-operandum procedure with reinforcement arranged according to concurrent variable-interval variable-interval schedules. Condition 1 involved choice between slides of women and slides of men; condition 2, slides of men vs. slides of children; and condition 3, slides of women vs. slides of children. On the assumption that the subject matched his ratios of responses and times on the two operanda to reinforcer value, estimates were obtained from the data in conditions 1 and 2 of the values of slides of women and of children relative to slides of men. These yielded predictions of the relative allocations of responses and of time in condition 3. The predicted and obtained outcomes were similar. The results support the use of matching-based scaling of reinforcer values in the prediction of choice between qualitatively different reinforcers and extend the application to a human subject, the two-operandum procedure, and sexual stimuli.     

Delayed and current stimulus control in successive discriminations

In a successive discrimination in which successively alternating red and green hues signaled component variable-interval schedules, sensitivity of the ratio of responses in the two components to variation in the component reinforcer ratio decreased systematically during the course of the component. This decrease in stimulus control or discrimination over the course of the component was shown to be the result of delayed control of responding during the component by the stimulus transition between components. When the red–green stimulus transition was altered by interpolating a white stimulus at the end of each 60-s component, discrimination at the beginning of the component (measured by the power-function exponent for sensitivity to reinforcement) was reduced. Conditions with the white stimulus inserted in other quarters of the component indicated that the current discriminative stimulus exerts control over responding throughout the component, whereas during about the first half of the component, response differentials are influenced by the transition between discriminative stimuli.