Session duration and the VI response function: Within-session prospective and retrospective effects

Two experiments examined the effects of session duration on responding during simple variable-interval schedules. In Experiment 1, rats were exposed to a series of simple variable-interval schedules differing in both session duration (10 min or 30 min) and scheduled reinforcement rate (7.5 s, 15 s, 30 s, and 480 s). The functions relating response rate to reinforcement rate were predominantly monotonic for the short (10-min) sessions but were predominantly bitonic for the long (30-min) sessions, when data from the entire session were considered. Examination of responding within sessions suggested that differences in the whole-session data were produced by a combination of prospective processes (i.e., processes based on events scheduled to occur later in the session) and retrospective processes (i.e., processes based on events that had already occurred in the session). In Experiment 2, rats were exposed to a modified discrimination procedure in which pellet flavor (standard or banana) predicted session duration (10 min or 30 min). All rats came to respond faster during the short (10-min) sessions than during the first 10 min of the long sessions. As in Experiment 1, the results seemed to reflect the simultaneous operation of both prospective and retrospective processes. The results shed light on the recent controversy over the form of the variable-interval response function by identifying one variable (session duration) and two types of processes (prospective and retrospective) that influence responding on these schedules.     

Elicited responding in chain schedules.

An omission procedure was employed to study elicited pecking in the first component of a two-component chain schedule. Both components were fixed-interval schedules correlated with colored keylights. The first response following the initial-link schedule produced a second fixed-interval schedule. We studied several fixed-interval lengths in two conditions: a standard response-dependent condition and an omission-contingent condition. The omission-contingent condition differed from the response-dependent condition in that responses during the initial fixed interval terminated the trial (omitting the terminal component and grain). If the terminal component was not omitted, a response following the terminal link's requirement produced 4-s access to grain. Pigeons responded during more than 70% of the initial links in the omission-contingent condition and responded during more than 90% of the initial links in the response-dependent condition. In general, rates of responding were consistent with the percentage data. The responding in the omission condition suggests that there may be elicited pecking, in chain schedules using pigeons, that is not the result of contingent conditioned reinforcement.     

The relationship between feeding rate and patch choice.

Rats in a laboratory foraging simulation searched for sequential opportunities to feed in two patches that differed in the rate at which food pellets were delivered (controlled by fixed-interval schedules) and in the size of the pellets. The profitability of feeding in each patch was calculated in terms of time (grams per minute) and in terms of effort (grams per bar press). These values were the result of the imposed fixed interval, the size of the pellets, and the rate at which the rats pressed the bar in each condition. The rats ate more food and larger meals, but not more frequent meals, at the patch offering the higher rate of food consumption, calculated as grams per minute. The relative intake at any patch was a function of the relative rate of intake during meals at that patch compared to the other patch. Rats respond to explicit manipulations of feeding time in the same manner as they respond to manipulations of feeding effort.     

Behavior-dependent reinforcer-rate changes in concurrent schedules: A further analysis

Six pigeons were trained on concurrent variable-interval schedules in three different procedures. The first procedure was a standard concurrent schedule, and the relative reinforcer frequency for responding was varied. The second was a schedule in which a relative left-key response rate (over a fixed period of time) exceeding .75 produced, in the next identical time period a higher reinforcer rate on the right key. If this criterion was not exceeded, equal reinforcer rates were arranged on the two keys in this period. This was the dependent procedure. In the third (independent) procedure, the periods of higher right-key reinforcer rates occurred with the same probability as in the second procedure, but occurred independently of behavior. In the second and third procedures, the fixed-time period (window) was varied from 5 s to 60 s, and to 240 s in the second procedure only. Performance on the two keys was similar in the concurrent and independent procedures. The procedure used in the dependent conditions generally affected performance when the windows were shorter than about 30 s. Models of performance that assume that subjects do not discriminate changes in local relative reinforcer rates cannot account for the data. Moreover, existing models are inherently unable to account for the effects of contingencies of reinforcement between responding on one alternative and gaining reinforcers on another that are arranged or that emerge as a result of time allocated to alternative schedules. Undermatching on concurrent variable-interval schedules may result from such emergent contingencies.     

Choice and conditioned reinforcement.

A potential weakness of one formulation of delay-reduction theory is its failure to include a term for rate of conditioned reinforcement, that is, the rate at which the terminal-link stimuli occur in concurrent-chains schedules. The present studies assessed whether or not rate of conditioned reinforcement has an independent effect upon choice. Pigeons responded on either modified concurrent-chains schedules or on comparable concurrent-tandem schedules. The initial link was shortened on only one of two concurrent-chains schedules and on only one of two corresponding concurrent-tandem schedules. This manipulation increased rate of conditioned reinforcement sharply in the chain but not in the tandem schedule. According to a formulation of delay-reduction theory, when the outcomes chosen (the terminal links) are equal, as in Experiment 1, choice should depend only on rate of primary reinforcement; thus, choice should be equivalent for the tandem and chain schedules despite a large difference in rate of conditioned reinforcement. When the outcomes chosen are unequal, however, as in Experiment 2, choice should depend upon both rate of primary reinforcement and relative signaled delay reduction; thus, larger preferences should occur in the chain than in the tandem schedules. These predictions were confirmed, suggesting that increasing the rate of conditioned reinforcement on concurrent-chains schedules may have no independent effect on choice.     

Response specificity in animal timing.

The stimuli that control responding in the peak procedure were investigated by training rats, in separate sessions, to make two different responses for food reinforcement. During one type of session, lever pressing was normally reinforced 32 s after the onset of a light. During the other type of session, chain pulling was normally reinforced either 8 s after the onset of one auditory cue or 128 s after the onset of a different auditory cue. For both types of sessions, only the appropriate manipulandum was available, and 20% of the trials lasted 240 s and involved no response-contingent consequences. Rats were then tested with the auditory cues in the presence of the lever and the light in the presence of the chain. If the time of reinforcement associated with each stimulus was learned, response rates should peak at these times during transfer testing. However, if a specific response pattern was learned for each stimulus, little transfer should occur. The results did not clearly support either prediction, leading to the conclusion that both a representation of the time of reinforcement and the rat's own behavior may control responding in this situation.     

Signal functions in delayed reinforcement

Three experiments were conducted with pigeons to examine the role of the signal in delay-of-reinforcement procedures. In the first, a blackout accompanying a period of nonreinforcement increased key-peck response rates maintained by immediate reinforcement. The effects of dissociating the blackout from the delay interval were examined in the second experiment. In three conditions, blackouts and unsignaled delays were negatively correlated or occurred randomly with respect to one another. A signaled delay and an unsignaled delay that omitted the blackouts were studied in two other conditions. All delay-of-reinforcement conditions generally produced response rates lower than those produced by immediate reinforcement. Signaled delays maintained higher response rates than did any of the various unsignaled-delay conditions, with or without dissociated blackouts. The effects of these latter conditions did not differ systematically from one another. The final experiment showed that response rates varied as a function of the frequency with which a blackout accompanied delay intervals. By eliminating a number of methodological difficulties present in previous delay-of-reinforcement experiments, these results suggest the importance of the signal in maintaining responding during delay-of-reinforcement procedures and, conversely, the importance of the delay interval in decreasing responding.     

Stimulus and reinforcer relativity in multiple schedules: Local and dimensional effects on sensitivity to reinforcement

Pigeons' responses in two successive components of multiple schedules were reinforced according to variable-interval schedules of reinforcement that varied over five different conditions. Within each session of all conditions, line orientations of 0°, 30°, or 45° in Component 1 alternated with orientations of 45°, 60°, or 90° in Component 2. Response rates were recorded in three successive subintervals of each component. Ratios were taken between the response rate in each Component 1 line orientation and the response rate in each Component 2 orientation. These ratios were found to be power functions of the corresponding ratios of obtained reinforcement rates. Sensitivity of response ratios to changes in reinforcer ratios, given by the value of the exponent of the power function, increased systematically with increasing disparity between the dimensional values of orientation stimuli. In addition, sensitivity decreased systematically over successive subintervals of components, that is, with increasing time since component alternation. Dimensional and local (subinterval) effects interacted in that sensitivity increased with stimulus disparity to a far greater extent in the first subinterval than later in components. The data could be described by a combination of rectangular hyperbolae which attributed the interaction between local and dimensional effects to limits set by local effects on the extent that stimulus differences could affect sensitivity.     

The effects of delayed reinforcement on free-operant responding

In previous studies of delayed reinforcement, response rate has been found to vary inversely with the response-reinforcer interval. However, in all of these studies the independent variable, response-reinforcer time, was confounded with the number of reinforcers presented in a fixed period of time (reinforcer frequency). In the present study, the frequency of available reinforcers was held constant, while temporal separation between response and reinforcer was independently manipulated. A repeating time cycle, T, was divided into two alternating time periods, t^D and t^Δ. The first response in t^D was reinforced at the end of the prevailing T cycle and extinction prevailed in t^Δ. Two placements for t^D were defined, an early t^D placement in which t^D precedes t^Δ and a late t^D placement in which t^D follows t^Δ. The duration of the early and late t^D was systematically decreased from 30 seconds (i.e., t^D = T) to 0.1 second. Manipulation of t^D placement and duration controlled the temporal separation between response and reinforcement, but it did not affect the frequency of programmed reinforcers, which was 1/T. The results show that early and late t^D placements of equal duration have similar overall effects upon response rate, reinforcer frequency, responses per reinforcer, and obtained response-reinforcer temporal separation. A stepwise regression analysis using log response rate as the dependent variable showed that the obtained delay was a significant first-step variable for six of eight subjects, with obtained reinforcer frequency significant for the remaining two subjects.     

Confirmation of linear system theory prediction: Changes in Herrnstein's k as a function of changes in reinforcer magnitude

Eight human subjects pressed a lever on a range of variable-interval schedules for 0.25¢ to 35.0¢ per reinforcement. Herrnstein's hyperbola described seven of the eight subjects' response-rate data well. For all subjects, the y-asymptote of the hyperbola increased with increasing reinforcer magnitude and its reciprocal was a linear function of the reciprocal of reinforcer magnitude. These results confirm predictions made by linear system theory; they contradict formal properties of Herrnstein's account and of six other mathematical accounts of single-alternative responding.