Changeover delay effects on topographically tagged discriminative behavior

In a multiple variable-interval extinction schedule, pigeons' responses on an operant key were differentially reinforced in the presence of discriminative stimuli located on a signal key. Changeover delays of zero, one, two, or four seconds specified the time following a signal-key response within which an operant-key response was not reinforced. Systematic reduction of signal-key response rates with increasing changeover-delay duration indicated that signal-key responding was largely maintained by reinforcement of adventitious signal-key/operant-key response chains.     

On the functions of the changeover delay

The function of changeover delays in producing matching was examined with pigeons responding on concurrent variable-interval variable-interval schedules. In Experiment 1, no changeover delay was compared to two different types of changeover delay. One type, designated generically as response-response but in the present example as peck-peck, was timed from the first response on the switched-to key; the other, designated generically as pause-response but in the present example as pause-peck, was timed from the last response on the switched-from key. High changeover rates occurred with no changeover delay. Peck-peck and pause-peck changeover delays produced low and intermediate changeover rates, respectively. In Experiment 2, pause-peck and peck-peck changeover delays were compared across a range of relative reinforcement rates. Similar matching relations developed despite differences in the changeover rates and local response patterns as a function of the type of changeover delay. In Experiment 3, both types of changeover delay yielded similar changeover rates when their obtained durations were equal via yoking. The results suggest that changeover delays function to separate responses on one key from reinforcers on the other or to delay reinforcement for changing over. In addition, the distribution of responding during and after the changeover delay may vary considerably without affecting matching.     

Changeover behavior and preference in concurrent schedules

Pigeons were trained on a multiple schedule of reinforcement in which separate concurrent schedules occurred in each of two components. Key pecking was reinforced with milo. During one component, a variable-interval 40-s schedule was concurrent with a variable-interval 20-s schedule; during the other component, a variable-interval 40-s schedule was concurrent with a variable-interval 80-s schedule. During probe tests, the stimuli correlated with the two variable-interval 40-s schedules were presented simultaneously to assess preference, measured by the relative response rates to the two stimuli. In Experiment 1, the concurrently available variable-interval 20-s schedule operated normally; that is, reinforcer availability was not signaled. Following this baseline training, relative response rate during the probes favored the variable-interval 40-s alternative that had been paired with the lower valued schedule (i.e., with the variable-interval 80-s schedule). In Experiment 2, a signal for reinforcer availability was added to the high-value alternative (i.e., to the variable-interval 20-s schedule), thus reducing the rate of key pecking maintained by that schedule but leaving the reinforcement rate unchanged. Following that baseline training, relative response rates during probes favored the variable-interval 40-s alternative that had been paired with the higher valued schedule. The reversal in the pattern of preference implies that the pattern of changeover behavior established during training, and not reinforcement rate, determined the preference patterns obtained on the probe tests.     

Levels of aggregation: Relative time allocation in concurrent-schedule performance

The proportion of time allocated to one component of a concurrent variable-interval variable-interval schedule was computed for groups of interchangeover times (aggregates) within several intact time series. Variability in obtained proportions decreased as the number of interchangeover times within each aggregate increased; however, modal proportions failed to correspond to overall relative time allocation computed over the course of an entire experimental session, even at the largest aggregate size. The aggregated time series showed periodicities at small aggregate sizes and general trends in local preference at larger aggregate sizes. It is suggested that overall relative time allocation represents a molar extreme in the aggregation of behavior that may not accurately reflect central tendency in the allocation of time to available alternatives within the context of ongoing behavior.     

Concurrent schedules: Spatial separation of response alternatives

Four pigeons were exposed to independent concurrent variable-interval 20-second variable-interval 60-second schedules of reinforcement. A transparent partition was inserted midway between the two response keys. The length of the partition was systematically manipulated. Increasing partition length produced a decrease in changeover rate in Experiment 1. Over-matching was observed with a partition length of 20 centimeters. In Experiment 2 a four-second limited hold was added to the schedules. Increasing partition length produced a decrease in changeover rate that exceeded the decrease observed in Experiment 1. This manipulation produced nearly exclusive choice of the variable-interval 20-second component. The present results, together with results obtained in related research, suggest that deviation from matching is a function of procedural variables that determine the consequences of a changeover response.     

Effects of the discriminability of alternatives in three-alternative concurrent-schedule performance

Six pigeons were trained on two- and three-alternative concurrent schedules in which the alternatives were signaled by different wavelengths of light on the main pecking key. The schedules were arranged according to a switching-key procedure in which pecks on a white side key produced a 3-s blackout and, intermittently, a change in the variable-interval schedule of food programmed on the main (center) key after the blackout. In Part 1, a two-alternative concurrent variable-interval schedule was arranged in which the alternatives were signaled by 560 nm and 630 nm. Parts 2 and 3 arranged three-alternative concurrent variable-interval schedules with the alternatives signaled by 560 nm, 600 nm, and 630 nm (Part 2) and 560 nm, 623 nm, and 630 nm (Part 3). Within each part, the relative rate of food reinforcers available on the alternatives was varied across a wide range. In all parts of the experiment, the ratios of responses emitted between pairs of alternatives were more extreme than the ratios of reinforcers obtained on the pairs of alternatives, a result termed overmatching. In Parts 2 and 3, generalized matching sensitivities between pairs of alternatives were found to be higher when the reinforcer rate on the third alternative was low than when it was high—an apparent failure of the constant-ratio rule. The data were well described by an extension of the Davison and Jenkins (1985) model, which assumes differing discriminabilities between concurrent-schedule alternatives in combination with a punishing effect of blackout following changeovers.     

Changeover ratio effects on concurrent variable-interval performance

Rats' bar-pressing was maintained by concurrent variable-interval schedules of reinforcement. A fixed-ratio of pulls on a chain (the changeover ratio) was required for switching between schedules. The first experiment employed equal variable-interval schedules and symmetrical changeover ratios. Increasing these ratios resulted in a decrease in the rate of switching between schedules and an increase in local response rate. In the second experiment, a range of asymmetrical changeover ratios was used with equal variable-interval schedules, and a preference was found for the schedule associated with the larger switching-into ratio. Both the distributions of responses and time between the two schedules deviated from those expected on the basis of obtained reinforcers. In the third experiment, the switching-out-of ratio was dependent on the amount of time spent in a variable-interval 2-minute schedule; a constant ratio permitted switching out of the alternative variable-interval 1-minute schedule. A strong preference was shown for the variable-interval 2-minute schedule. The fourth experiment used equal variable-interval schedules; one changeover ratio was varied while the second remained constant. The results failed to show systematic differences in local response rates immediately after a changeover.     

Open versus closed economies: performance of domestic hens under fixed ratio schedules

Domestic hens responded under fixed-ratio schedules of food (wheat) reinforcement under several experimental conditions. Part 1 (open economy) investigated performance on fixed-ratio schedules over both multisession steady-state conditions and daily changes of the schedule, with hens maintained at 80% of free-feeding weights by extraexperimental feeding. In Parts 2 and 3 (closed economy and short sessions) sessions were 40 min long, and the hens' weights were allowed to vary (Part 2) or sessions were conducted only when the hens were at approximately 80% of free-feeding weights (Part 3). In Part 4 (closed economy and long sessions) sessions were 24 hr long and the fixed-ratio requirement was changed either daily or after 7 consecutive days. In general, the daily changes of fixed-ratio requirement in the open economy and short-session closed economy gave much the same result as the steady-state open-economy sessions. Overall response and reinforcer rates decreased with increasing fixed-ratio requirement (except at the shortest fixed ratios). Running response rates decreased, and postreinforcement pauses generally increased. In contrast, overall response rates in the long-session closed economy generally increased with the fixed-ratio requirement. Session length is suggested as a cause of the differences between the short- and long-session closed-economy results.     

Choice, changing over, and reinforcement delays

In three experiments, pigeons were used to examine the independent effects of two normally confounded delays to reinforcement associated with changing between concurrently available variable-interval schedules of reinforcement. In Experiments 1 and 2, combinations of changeover-delay durations and fixed-interval travel requirements were arranged in a changeover-key procedure. The delay from a changeover-produced stimulus change to a reinforcer was varied while the delay between the last response on one alternative and a reinforcer on the other (the total obtained delay) was held constant. Changeover rates decreased as a negative power function of the total obtained delay. The delay between a changeover-produced stimulus change had a small and inconsistent effect on changeover rates. In Experiment 3, changeover delays and fixed-interval travel requirements were arranged independently. Changeover rates decreased as a negative power function of the total obtained delay despite variations in the delay from a change in stimulus conditions to a reinforcer. Periods of high-rate responding following a changeover, however, were higher near the end of the delay from a change in stimulus conditions to a reinforcer. The results of these experiments suggest that the effects of changeover delays and travel requirements primarily result from changes in the delay between a response at one alternative and a reinforcer at the other, but the pattern of responding immediately after a changeover depends on the delay from a changeover-produced change in stimulus conditions to a reinforcer.     

Sensitivity of time allocation to concurrent-schedule reinforcement

Four pigeons were trained on concurrent variable-interval schedules programmed on a center response key, with access to those schedules controlled by responses on left or right side keys. Two procedures were used. In one, the pigeon was given limited access, in that each side-key response produced 3-s access to a center-key schedule, and in the other procedure, access was unlimited. Data were analyzed using the generalized matching law. Comparison of sensitivities to reinforcement of interchangeover time for both procedures showed them to be of similar magnitude. Response sensitivities were also similar in magnitude for both procedures. From the limited-access procedure a second time measure that was available, switched-in time, was relatively uncontaminated by time spent emitting behavior other than key pecking. Sensitivities to reinforcement for the switched-in time measure were always smaller than interchangeover-time sensitivities for either procedure, and were approximately equal to response sensitivities for the limited-access procedure. Two other access times (5 and 7.5 s) were studied to validate the choice of 3 s as the main access time. These results indicate that when time spent emitting other behavior is excluded from interchangeover time, time and response sensitivities will be approximately equal.     

Travel time and concurrent-schedule choice: retrospective versus prospective control

Six pigeons were trained on concurrent variable-interval schedules in which two different travel times between alternatives, 4.5 and 0.5 s, were randomly arranged. In Part 1, the next travel time was signaled while the subjects were responding on each alternative. Generalized matching analyses of performance in the presence of the two travel-time signals showed significantly higher response and time sensitivity when the longer travel time was signaled compared to when the shorter time was signaled. When the data were analyzed as a function of the previous travel time, there were no differences in sensitivity. Dwell times on the alternatives were consistently longer in the presence of the stimulus that signaled the longer travel time than they were in the presence of the stimulus that signaled the shorter travel time. These results are in accord with a recent quantitative account of the effects of travel time. In Part 2, no signals indicating the next travel time were given. When these data were analyzed as a function of the previous travel time, time-allocation sensitivity after the 4.5-s travel time was significantly greater than that after the 0.5-s travel time, but no such difference was found for response allocation. Dwell times were also longer when the previous travel time had been longer.     

Preference for mixed versus constant delay of reinforcement

Preference for constant and mixed delay of reinforcement was studied using concurrent equal variable-interval schedules. For four pigeons, pecking one key was reinforced following constant delays of 8 sec and mixed delays of 6 or 10 and 2 or 14 sec. Pecking a second key was reinforced following constant delays of 0, 8, 16, and 32 sec. For two additional pigeons, pecking one key was reinforced following delays of 30, 15 or 45, 5 or 55, and 0 or 60 sec. Reinforcements on the other key were delayed 30 sec. It was found that (a) pigeons preferred mixed relative to constant delay of reinforcement, and (b) preference for mixed delay of reinforcement increased as the range of delay interval variability increased.     

Choice performance in several concurrent key-peck treadle-press reinforcement schedules

Five pigeons were exposed to several concurrent variable-interval food reinforcement schedules. For three subjects, one component of the schedule required a key-pecking response, the other a treadle-pressing response. For the other two subjects, both schedule components required treadle-pressing responses. The relative probability of reinforcement associated with the manipulanda was varied from 0 to 1.0 in 13 experimental conditions for the Key-Treadle subjects and nine conditions for the Treadle-Treadle subjects. The results indicated that the logarithms of relative time spent responding, and the logarithms of relative number of responses emitted on a manipulandum, approximated direct linear functions of logarithms of the relative frequencies of reinforcement associated with that manipulandum. No systematic bias in favor of time spent key pecking over time spent treadle pressing was apparent for the Key-Treadle subjects. All subjects exhibited undermatching, in that the ratios of time and response allocation at the alternatives systematically differed from the ratios of reinforcers obtained from the alternatives in the direction of indifference. Key pecking appeared to have no special link to food beyond treadle pressing or what would be expected on the basis of the reinforcement dependencies alone.     

Choice, experience, and the generalized matching law

Five pigeons were exposed to different pairs of concurrent variable-interval, variable-interval schedules on nine experimental conditions of 30 sessions each. For every session, the parameters of the generalized matching equation were computed for the first five, six, seven, eight, and nine experimental conditions. The exponent a, both for response and time distribution, tended to decrease with increases in number of experimental conditions and to increase with number of sessions per condition, but values of k (bias) varied unsystematically. When the subjects were exposed to five new pairs of schedules, with 55 sessions per condition, the findings were confirmed. Data from the literature on the generalized matching law suggest that the variability of exponent values may be explained in part by the use of naive or experienced subjects in different investigations and by the variability in number of experimental conditions and in number of sessions per condition.     

Effects of varying stimulus disparity and the reinforcer ratio in concurrent-schedule and signal-detection procedures.

The present study measured the effects of stimulus and reinforcer variations on pigeons' behavior in two different choice procedures. Two intensities of white light were presented as the stimuli on the main key in a switching-key concurrent schedule and as the sample stimuli in a signal-detection procedure. Under both procedures, the scheduled rate of reinforcement was varied across conditions to produce various ratios of obtained reinforcement. These ratios were obtained for seven pairs of light intensities. In the concurrent schedules, the effects of reinforcer-ratio variations were positively correlated with the physical disparity between the two light intensities. In the signal-detection procedure, changes in the reinforcer ratio produced greater effects on performance when stimulus disparity was very low or very high compared to those found at intermediate levels of stimulus disparity. This discrepancy creates a dilemma for existing behavioral models of signal-detection performance.     

Concurrent schedules: Quantifying the aversiveness of noise

Four hens worked under independent multiple concurrent variable-interval schedules with an overlaid aversive stimulus (sound of hens in a poultry shed at 100dBA) activated by the first peck on a key. The sound remained on until a response was made on the other key. The key that activated the sound in each component was varied over a series of conditions. When the sound was activated by the left (or right) key in one component, it was activated by the right (or left) key in the other component. Bias was examined under a range of different variable-interval schedules, and the applicability of the generalized matching law was examined. It was found that the hens' behavior was biased away from the sound independently of the schedule in effect and that this bias could be quantified using a modified version of the generalized matching law. Behavior during the changeover delays was not affected by the presence of the noise or by changes in reinforcement rate, even though the total response measures were. Insensitivity shown during the delay suggests that behavior after the changeover delay may be more appropriate as a measure of preference (or aversiveness) of stimuli than are overall behavior measures.     

Time allocation in concurrent schedules: the effect of signalled reinforcement

The responses of five pigeons were reinforced on concurrent variable-interval variable-interval reinforcement schedules in which changeover key responses changed the stimulus and reinforcement schedules associated with the food key. While the reinforcement availability in one component remained unchanged throughout the experiment, the reinforcement availability in the other component was, during several conditions, signalled by the onset of an additional discriminative stimulus. During unsignalled conditions, both the relative frequency of responding and the relative time spent in each component approximated the obtained relative reinforcement frequency in each component. The effect of signalling reinforcer availability in one component was to (1) reduce responding in the signalled component to near-zero levels, and (2) increase the relative time in the unsignalled component, without a corresponding increase in the obtained relative reinforcement frequency. The magnitude of the increase in relative time in the unsignalled component decreased as the overall frequency of reinforcement increased. This deviation in the matching relation between relative time and the obtained relative reinforcement frequency was eliminated if the overall reinforcement frequency was increased before the signal was introduced and then, without removing the signal, gradually reduced.