Within-session Changes In Responding During Concurrent Variable-interval Schedules

Five rats and 4 pigeons responded for food delivered by several concurrent variable-interval schedules. The sum of the rates of reinforcement programmed for the two components varied from 15 to 480 reinforcers per hour in different conditions. Rates of responding usually changed within the experimental session in a similar manner for the two components of each concurrent schedule. The within-session changes were similar to previously reported changes during simple schedules that provided rates of reinforcement equal to the sum of all reinforcers obtained from the concurrent schedules. The number of changeovers also changed within sessions in a manner similar to the changes in instrumental responding. These results suggest that changeovers are governed by the same variables that govern instrumental responding. They also suggest that the within-session change in responding during each component of a concurrent schedule is determined by approximately the sum of the reinforcers obtained from both components when both components provide the same type of reinforcer.     

Behavioral economics and within-session changes in responding

Pigeons and rats responded on fixed-ratio schedules with requirements ranging from 5 to 120 responses. Consistent with past results from several schedules and procedures, responding usually changed systematically within experimental sessions. The within-session changes were usually larger and were less symmetrical around the middle of the session for schedules that provided higher, rather than lower, rates of reinforcement. These results suggest that similar variables contribute to within-session changes in responding under different schedules. When an economic demand function was fit to the data, the intensity and elasticity of demand for food and the percentage of the variance accounted for decreased within sessions, although the trend for elasticity did not reach statistical significance for pigeons. These results suggest that relatively short sessions should be used to study economic demand in open economies and that demand may differ at different times in a session and in sessions of different lengths. Within-session changes in intensity, but not necessarily elasticity, of demand are consistent with behavioral economic theories.     

Within-session changes in responding during several simple schedules

Pigeons' key pecking was reinforced by food delivered by several fixed-interval, variable-ratio, and differential-reinforcement-of-low-rate schedules. Rate of responding, number of responses per reinforcer, length of postreinforcement pause, running response rate, and the time required to collect an available reinforcer changed systematically within sessions when the schedules provided high rates of reinforcement, but usually not when they provided low rates. These results suggest that the factors that produce within-session changes in responding are generally similar for different schedules of reinforcement. However, a separate factor may also contribute during variable-ratio schedules. The results question explanations for within-session changes that are related solely to the passage of time, to responding, and to one interpretation of attention. They support the idea that one or more factors related to reinforcement play a role.     

Within-session Response Patterns On Conjoint Variable-interval Variable-time Schedules

Operant responding often changes within sessions, even when factors such as rate of reinforcement remain constant. The present study was designed to determine whether within-session response patterns are determined by the total number of reinforcers delivered during the session or only by the reinforcers earned by the operant response. Four rats pressed a lever and 3 pigeons pecked a key for food reinforcers delivered by a conjoint variable-interval variable-time schedule. The overall rate of reinforcement of the conjoint schedule varied across conditions from 15 to 480 reinforcers per hour. When fewer than 120 reinforcers were delivered per hour, the within-session patterns of responding on conjoint schedules were similar to those previously observed when subjects received the same total number of reinforcers by responding on simple variable-interval schedules. Response patterns were less similar to those observed on simple variable-interval schedules when the overall rate of reinforcement exceeded 120 reinforcers per hour. These results suggest that response-independent reinforcers can affect the within-session pattern of responding on a response-dependent schedule. The results are incompatible with a response-based explanation of within-session changes in responding (e.g., fatigue), but are consistent with both reinforcer-based (e.g., satiation) and stimulus-based (e.g., habituation) explanations.     

Within-session changes in key and lever pressing for water during several multiple variable-interval schedules

Rats pressed keys or levers for water reinforcers delivered by several multiple variable-interval schedules. The programmed rate of reinforcement varied from 15 to 240 reinforcers per hour in different conditions. Responding usually increased and then decreased within experimental sessions. As for food reinforcers, the within-session changes in both lever and key pressing were smaller, peaked later, and were more symmetrical around the middle of the session for lower than for higher rates of reinforcement. When schedules provided high rates of reinforcement, some quantitative differences appeared in the within-session changes for lever and key pressing and for food and water. These results imply that basically similar factors produce within-session changes in responding for lever and key pressing and for food and water. The nature of the reinforcer and the choice of response can also influence the quantitative properties of within-session changes at high rates of reinforcement. Finally, the results show that the application of Herrnstein's (1970) equation to rates of responding averaged over the session requires careful consideration.     

Choice performance in several concurrent key-peck treadle-press reinforcement schedules

Five pigeons were exposed to several concurrent variable-interval food reinforcement schedules. For three subjects, one component of the schedule required a key-pecking response, the other a treadle-pressing response. For the other two subjects, both schedule components required treadle-pressing responses. The relative probability of reinforcement associated with the manipulanda was varied from 0 to 1.0 in 13 experimental conditions for the Key-Treadle subjects and nine conditions for the Treadle-Treadle subjects. The results indicated that the logarithms of relative time spent responding, and the logarithms of relative number of responses emitted on a manipulandum, approximated direct linear functions of logarithms of the relative frequencies of reinforcement associated with that manipulandum. No systematic bias in favor of time spent key pecking over time spent treadle pressing was apparent for the Key-Treadle subjects. All subjects exhibited undermatching, in that the ratios of time and response allocation at the alternatives systematically differed from the ratios of reinforcers obtained from the alternatives in the direction of indifference. Key pecking appeared to have no special link to food beyond treadle pressing or what would be expected on the basis of the reinforcement dependencies alone.     

Matching, contrast, and equalizing in the concurrent lever-press responding of rats

Five rats pressed levers for food reinforces delivered by several concurrent variable-interval variable-interval schedules. The rate of reinforcement available for responding on one component schedule was held constant at 60 reinforcers per hour. The rate of reinforcement available for responding on the other schedule varied from 30 to 240 reinforcers per hour. The behavior of the rats resembled the behavior of pigeons pecking keys for food reinforcers. The ratio of the overall rates of responding emitted under, and the ratio of the time spent responding under, the two components of each concurrent schedule were approximately equal to the ratio of the overall rates of reinforcement obtained from the components. The overall rate of responding emitted under, and the time spent responding under, the variable component schedule varied directly with the overall rate of reinforcement from that schedule. The overall rate of responding emitted under, and the time spent responding under, the constant component schedule varied inversely with the overall rate of reinforcement obtained from the variable component. The local rates of responding emitted under, and the local rates of reinforcement obtained from, the two components did not differ consistently across subjects. But they were not exactly equal either.     

Earning and obtaining reinforcers under concurrent interval scheduling

Contingencies of reinforcement specify how reinforcers are earned and how they are obtained. Ratio contingencies specify the number of responses that earn a reinforcer, and the response satisfying the ratio requirement obtains the earned reinforcer. Simple interval schedules specify that a certain time earns a reinforcer, which is obtained by the first response after the interval. The earning of reinforcers has been overlooked, perhaps because simple schedules confound the rates of earning reinforcers with the rates of obtaining reinforcers. In concurrent variable-interval schedules, however, spending time at one alternative earns reinforcers not only at that alternative, but at the other alternative as well. Reinforcers earned for delivery at the other alternative are obtained after changing over. Thus the rates of earning reinforcers are not confounded with the rate of obtaining reinforcers, but the rates of earning reinforcers are the same at both alternatives, which masks their possibly differing effects on preference. Two experiments examined the separate effects of earning reinforcers and of obtaining reinforcers on preference by using concurrent interval schedules composed of two pairs of stay and switch schedules (MacDonall, 2000). In both experiments, the generalized matching law, which is based on rates of obtaining reinforcers, described responding only when rates of earning reinforcers were the same at each alternative. An equation that included both the ratio of the rates of obtaining reinforcers and the ratio of the rates of earning reinforcers described the results from all conditions from each experiment.     

Choice and the rate of punishment in concurrent schedules

Rats' responses on two levers were reinforced according to independent random-interval 1.5-min food schedules. In addition, both lever presses were intermittently punished according to several concurrent random-interval random-interval shock schedules. For the left, the scheduled rate of punishment was kept constant according to a random-interval 6-min schedule. For the right, the rate of punishment varied. As the frequency of punishment for the right lever press increased, its rate decreased. The rate of the left punished lever press increased, however, even though its scheduled reinforcement rate and punishment rate remained unchanged.     

Dynamical concurrent schedules

Previous work on the matching law has predominantly focused on the molar effects of the contingency by examining only one reinforcer ratio for extended periods. Responses are distributed as a function of reinforcer ratios under these static conditions. But the outcome under a dynamic condition in which reinforcer ratios change continuously has not been determined. The present study implemented concurrent variable-interval schedules that changed continuously across a fixed 5-min trial. The schedules were reciprocally interlocked. The variable interval for one key changed continuously from a variable-interval 15-s to a variable-interval 480-s, while the schedule for the other key changed from a variable-interval 480-s to a variable-interval 15-s. This dynamical concurrent schedule shifted behavior in the direction of matching response ratios to reinforcer ratios. Sensitivities derived from the generalized matching law were approximately 0.62, the mean absolute bias was approximately 0.11, and r2s were approximately 0.86. It was concluded that choice behavior can come to adapt to reinforcer ratios that change continuously over a relatively short time and that this change does not require extensive experience with a fixed reinforcer ratio. The results were seen as supportive of the view that all behavior constitutes choice.     

Changeover behavior and preference in concurrent schedules

Pigeons were trained on a multiple schedule of reinforcement in which separate concurrent schedules occurred in each of two components. Key pecking was reinforced with milo. During one component, a variable-interval 40-s schedule was concurrent with a variable-interval 20-s schedule; during the other component, a variable-interval 40-s schedule was concurrent with a variable-interval 80-s schedule. During probe tests, the stimuli correlated with the two variable-interval 40-s schedules were presented simultaneously to assess preference, measured by the relative response rates to the two stimuli. In Experiment 1, the concurrently available variable-interval 20-s schedule operated normally; that is, reinforcer availability was not signaled. Following this baseline training, relative response rate during the probes favored the variable-interval 40-s alternative that had been paired with the lower valued schedule (i.e., with the variable-interval 80-s schedule). In Experiment 2, a signal for reinforcer availability was added to the high-value alternative (i.e., to the variable-interval 20-s schedule), thus reducing the rate of key pecking maintained by that schedule but leaving the reinforcement rate unchanged. Following that baseline training, relative response rates during probes favored the variable-interval 40-s alternative that had been paired with the higher valued schedule. The reversal in the pattern of preference implies that the pattern of changeover behavior established during training, and not reinforcement rate, determined the preference patterns obtained on the probe tests.     

Matching and maximizing with concurrent ratio-interval schedules.

Animals exposed to standard concurrent variable-ratio variable-interval schedules could maximize overall reinforcement rate if, in responding, they showed a strong response bias toward the variable-ratio schedule. Tests with the standard schedules have failed to find such a bias and have been widely cited as evidence against maximization as an explanation of animal choice behavior. However, those experiments were confounded in that the value of leisure (behavior other than the instrumental response) partially offsets the value of reinforcement. The present experiment provides another such test using a concurrent procedure in which the confounding effects of leisure were mostly eliminated while the critical aspects of the concurrent variable-ratio variable-interval contingency were maintained: Responding in one component advanced only its ratio schedule while responding in the other component advanced both ratio schedules. The bias toward the latter component predicted by maximization theory was found.     

The effects of concurrent responding and reinforcement on behavioral output

Four birds key pecked on concurrent variable-interval one-minute variable-interval four-minute schedules with a two-second changeover delay. Response rates to the variable-interval one-minute key were then reduced by signaling its reinforcer availability and later by providing its reinforcers independently of responding. Each manipulation increased response rates to the variable-interval four-minute key even though relative reinforcement rates were unchanged. In a final phase, eliminating the variable-interval one-minute key and its schedule produced the highest rates of all to the variable-interval four-minute key. These results show that both reinforcement and response rates to one schedule influence response rates to another schedule. These results join those of Guilkey, Shull, & Brownstein (1975) in failing to replicate Catania (1963). Moreover, they violate the predictions of the equation for simple action (de Villiers & Herrnstein, 1976). In terms of a median-rate measure (reciprocal of the median interresponse time), rates to the variable-interval four-minute key were high when responding was not reduced to the variable-interval one-minute key and were low when it was reduced. This rate difference suggests a process difference between concurrent-schedule procedures that maintain high concurrent response rates versus those that do not. This process difference jeopardizes attempts to integrate single- and concurrent-operant performances within a single formulation.     

Within-session Changes In Responding During Autoshaping And Automaintenance Procedures

Four pigeons were exposed to autoshaping procedures in which an 8-second light on a response key was followed by food. Pecks on the key had no scheduled consequences. Subjects were also exposed to negative automaintenance procedures in which a peck on the illuminated key canceled the following food. The intertrial interval varied from an average of 7 seconds to an average of 232 seconds in different conditions. Rate of responding usually changed within sessions during autoshaping. Responding also changed within sessions for the 1 subject that responded during negative automaintenance. The within-session patterns of responding were flatter, peaked later, and were more symmetrical around the middle of the sessions at lower rates of food presentation, regardless of whether subjects responded on autoshaping, negative automaintenance, or previously reported variable-interval schedules. These results imply that similar variables produce within-session changes in responding during both classical (Pavlovian) and operant conditioning procedures.     

Timeout from concurrent schedules.

Response-contingent timeouts of equal duration and frequency were added to both alternatives of unequal concurrent schedules of reinforcement. For each of 4 pigeons in Experiment 1, relative response rates generally became less extreme as the frequency of timeout increased. In Experiment 2, relative response rates consistently approached indifference as the duration of timeout was increased. Variation in time allocation was less consistent in both experiments. Absolute response rates did not vary with the timeout contingency in either experiment. In a third experiment, neither measure of choice varied systematically when the duration of a postreinforcement blackout was varied. In contrast to the present results, preference has been shown to vary directly with the parameters of shock delivery in related procedures. The pattern of results in the first two experiments follows that obtained with other manipulations of the overall rate of reinforcement in concurrent schedules. The results of the third experiment suggest that an intertrial interval following reinforcement is not a critical feature of the overall rate of reinforcement.     

Residence time and choice in concurrent foraging schedules

Five pigeons were trained on a concurrent-schedule analogue of the “some patches are empty” procedure. Two concurrently available alternatives were arranged on a single response key and were signaled by red and green keylights. A subject could travel between these alternatives by responding on a second yellow “switching” key. Following a changeover to a patch, there was a probability (p) that a single reinforcer would be available on that alternative for a response after a time determined by the value of λ, a probability of reinforcement per second. The overall scheduling of reinforcers on the two alternatives was arranged nonindependently, and the available alternative was switched after each reinforcer. In Part 1 of the experiment, the probabilities of reinforcement, ρ_red and ρ_green, were equal on the two alternatives, and the arranged arrival rates of reinforcers, λ_red and λ_green, were varied across conditions. In Part 2, the reinforcer arrival times were arranged to be equal, and the reinforcer probabilities were varied across conditions. In Part 3, both parameters were varied. The results replicated those seen in studies that have investigated time allocation in a single patch: Both response and time allocation to an alternative increased with decreasing values of λ and with increasing values of ρ, and residence times were consistently greater than those that would maximize obtained reinforcer rates. Furthermore, both response- and time-allocation ratios undermatched mean reinforcer-arrival time and reinforcer-frequency ratios.     

Behavior-dependent reinforcer-rate changes in concurrent schedules: A further analysis

Six pigeons were trained on concurrent variable-interval schedules in three different procedures. The first procedure was a standard concurrent schedule, and the relative reinforcer frequency for responding was varied. The second was a schedule in which a relative left-key response rate (over a fixed period of time) exceeding .75 produced, in the next identical time period a higher reinforcer rate on the right key. If this criterion was not exceeded, equal reinforcer rates were arranged on the two keys in this period. This was the dependent procedure. In the third (independent) procedure, the periods of higher right-key reinforcer rates occurred with the same probability as in the second procedure, but occurred independently of behavior. In the second and third procedures, the fixed-time period (window) was varied from 5 s to 60 s, and to 240 s in the second procedure only. Performance on the two keys was similar in the concurrent and independent procedures. The procedure used in the dependent conditions generally affected performance when the windows were shorter than about 30 s. Models of performance that assume that subjects do not discriminate changes in local relative reinforcer rates cannot account for the data. Moreover, existing models are inherently unable to account for the effects of contingencies of reinforcement between responding on one alternative and gaining reinforcers on another that are arranged or that emerge as a result of time allocated to alternative schedules. Undermatching on concurrent variable-interval schedules may result from such emergent contingencies.     

Matching in concurrent variable-interval avoidance schedules

After pretraining with multiple variable-interval avoidance schedules, two rats were exposed to a series of concurrent variable-interval avoidance schedules. Responses on two levers cancelled delivery of electric shocks arranged according to two independent variable-interval schedules. The ratio of responses and time spent on the two levers approximately matched the ratio of shocks avoided on each. Matching to the number of shocks received was not obtained. Concurrent variable-interval avoidance can therefore be added to the group of positive and negative reinforcement schedules that can be expressed in the quantitative framework of the matching law.     

Patterns of responding within sessions.

Rates of responding changed systematically across sessions for rats pressing levers and keys and for pigeons pressing treadles and pecking keys. A bitonic function in which response rates increased and then decreased across sessions was the most common finding, although an increase in responding also occurred alone. The change in response rate was usually large. The function relating responding to time in session had the following general characteristics: It appeared early in training, and further experience moved and reduced its peak; it was flatter for longer sessions; and it was flatter, more symmetrical, and peaked later for lower than for higher rates of reinforcement. Factors related to reinforcement exerted more control over the location of the peak rate of responding and the steepness of the decline in response rates than did factors related to responding. These within-session changes in response rates have fundamental theoretical and methodological implications.