Second-order schedules of token reinforcement: comparisons of performance under fixed-ratio and variable-ratio exchange schedules

Rats' lever pressing produced tokens according to a 20-response fixed-ratio schedule. Sequences of token schedules were reinforced under a second-order schedule by presentation of periods when tokens could be exchanged for food pellets. When the exchange period schedule was a six-response fixed ratio, patterns of completing the component token schedules were bivalued, with relatively long and frequent pauses marking the initiation of each new sequence. Altering the exchange period schedule to a six-response variable ratio resulted in sharp reductions in the frequency and duration of these initial pauses, and increases in overall rates of lever pressing. These results are comparable to those ordinarily obtained under simple fixed-ratio and variable-ratio schedules.     

The effect of foreperiod duration on reaction time and its relation to interval schedules of reinforcement

Two groups of pigeons were exposed to a simple reaction-time procedure in which mean foreperiod duration was 5, 10, or 20 seconds. For one group, the foreperiods had an arithmetic, or rectangular, distribution; for the second group, they had a constant-probability, or Bernoulli, distribution. Under both distributions, mean response latency was an increasing, negatively accelerated function of mean foreperiod duration. On a given trial, response latency was a function of its associated foreperiod duration: latency was a decreasing function of foreperiod duration in the arithmetic distribution, and an increasing function of foreperiod duration in the constant-probability distribution. Examination of the distribution of latencies revealed a harmonic structure reminiscent of distributions of interresponse times under variable-interval schedules of reinforcement. Taken together, the results confirm and extend previous findings with human subjects, and also suggest numerous similarities to behavior maintained by variable-interval schedules.     

Changeover delay effects on topographically tagged discriminative behavior

In a multiple variable-interval extinction schedule, pigeons' responses on an operant key were differentially reinforced in the presence of discriminative stimuli located on a signal key. Changeover delays of zero, one, two, or four seconds specified the time following a signal-key response within which an operant-key response was not reinforced. Systematic reduction of signal-key response rates with increasing changeover-delay duration indicated that signal-key responding was largely maintained by reinforcement of adventitious signal-key/operant-key response chains.     

On the analysis of studies of choice

In a review of 103 sets of data from 23 different studies of choice, Baum (1979) concluded that whereas undermatching was most commonly observed for responses, the time measure generally conformed to the matching relation. A reexamination of the evidence presented by Baum concludes that undermatching is the most commonly observed finding for both measures. Use of the coefficient of determination by both Baum (1979) and de Villiers (1977) for assessing when matching occurs is criticized on statistical grounds. An alternative to the loss-in-predictability criterion used by Baum (1979) is proposed. This alternative statistic has a simple operational meaning and is related to the usual F-ratio test. It can therefore be used as a formal test of the hypothesis that matching occurs. Baum (1979) also suggests that slope values of between .90 and 1.11 can be considered good approximations to matching. It is argued that the establishment of a fixed interval as a criterion for determining when matching occurs, is inappropriate. A confidence interval based on the data from any given experiment is suggested as a more useful method of assessment.     

An invariant relation between changing over and reinforcement

Although concurrent schedules may arrange reinforcers irregularly, relatively large numbers of reinforcers are obtained when an animal changes from one schedule to the other. This paper proposes a quantitative relation that predicts the proportion of reinforcers obtained when an animal is working on a schedule and the proportion when the animal changes over to a schedule. Basically the relation states that the number of reinforcers obtained while an animal works on a schedule varies directly with the relative amount of time spent working on that schedule; and the number of reinforcers obtained when an animal changes to a schedule varies directly with the relative amount of time spent on the alternate schedule. An important aspect of this relation is that when relative reinforcement rates are less than .50, more reinforcers are obtained just after an animal changes to a schedule than at all other times when this schedule is engaged. Data obtained both from a stat-bird and a live pigeon were in close agreement with the quantitative predictions. The relation between changing over and reinforcement held across several procedural changes that included changes in relative reinforcement rate, changes from independent to interdependent scheduling procedures, and changes in the variable-interval reinforcement distributions. The results are discussed in terms of the effects of the local distribution of reinforcement on responding. The local reinforcement distribution can affect local response rates and affects the resulting matching relation. This arrangement has implications for explanations of choice.     

Reinforcing the absence of fixed-ratio performance

Pigeons received food for key pecking according to a fixed-ratio schedule, while, at the same time, food also was available for not pecking for a specified time. With a fixed ratio of 60, responding was not affected by not-pecking times of 80 or 40 seconds, and was eliminated completely at 10 seconds. With ratios of 180, pecking stopped with not-pecking times of 80 seconds or less; with ratios of 300, it stopped at 120 seconds or less. Not-responding schedules produced steady-state performance immediately following contact with the schedule. With return to the fixed-ratio schedule alone, response rate sometimes was elevated temporarily. When response-independent food presentations replaced the not-pecking schedule, response rate often was enhanced, and the ratio pattern was lost. Only the highest densities of food delivery eliminated responding, even with a fixed ratio of 300. In general, the effects corresponded to those of punishment, except that contrast had appeared both during and after punishment, and now appeared only after the response elimination procedure was suspended.     

Periodicities within a fixed-interval session

Within-session periodicities in number of responses per interval and postreinforcement pauses were investigated on fixed-interval schedules of 1, 2, and 3 minutes with rats. Postreinforcement pause values and the number of responses in successive intervals were not systematically related. The direction of change of these variables from one pair of intervals to the next revealed periodicities in that the direction of change varied more than would be expected by chance. A response prevention technique used to manipulate the length of time spent responding in an interval had little effect on the postreinforcement pause value of the next interval except when only a single response was permitted in an interval. This procedure tended to reduce the postreinforcement pause value of the next interval to an abnormally low level.     

Response requirements as constraints on output

Two experiments studied how added response requirements affected fixed-interval schedule performance. Experiment 1 involved tandem fixed-interval fixed-ratio schedules, and Experiment 2 studied conjunctive fixed-interval fixed-ratio schedules. In both, pigeons' output, defined as overall response rate or as responses during the interval, first increased and then decreased as the ratio was raised. With small ratio requirements, the frequency of reinforcement in time either did not change or decreased slightly. With progressively larger ratios, reinforcement frequency decreased consistently. Alternative explanations were discussed. The first, a reinforcement theory account, was that response strength is an increasing monotonic function of both the response requirement and reinforcement frequency, and the bitonic output function represents interacting effects. Increases in the response requirement accompanied by small changes in reinforcement frequency enhance output, but further increases result in large enough decrements in reinforcement frequency so that output is lowered. The second explanation does not view reinforcement as a basic process but, instead, derives from concepts of economics and conservation. Organisms allocate their behavior among alternatives so as to maximize value, where value is a function of the responses that can occur in a given situation under the set of restrictions imposed by particular schedules. One form of this theory explicitly predicts that output is a bitonic function of ratio requirements in simple ratio schedules. However, it was not clear that this model could explain the present effects involving joint ratio and interval schedule restrictions.     

Free-operant choice behavior: A molecular analysis

Pigeons' pecks to two concurrent initial-link stimuli occasionally produced one of two mutually exclusive terminal links. Further responding to the terminal-link stimulus produced food under fixed-interval or fixed-ratio schedules. In such concurrent chained schedules, investigators rarely use a changeover delay to control superstitious switching, although it is customary to use a changeover delay in simple concurrent schedules in which choice responses produce food directly. When terminal-link fixed-interval schedules were equal or similar in duration and no changeover delay was employed, conditional probabilities of choice for a schedule were found to be lower if the last choice was for that schedule than if the last choice was for the other schedule (“switching dependency”). Imposition of a changeover delay with equal or unequal terminal links produced the opposite pattern: conditional probabilities of choice for a schedule were higher if the last choice was for that schedule than if the last choice was for the other schedule. Turning off all chamber lights during the changeover delay interval attenuated these “repetition dependencies.” The results indicate that excessive switching can complicate the interpretation of data from concurrent chains much as from simple concurrent schedules, and that using blackouts to control switching may be preferable to using a changeover delay.     

DEVIATIONS FROM MATCHING AS A MEASURE OF PREFERENCE FOR ALTERNATIVES IN PIGEONS

Preferences for larger or smaller formally defined response classes were investigated in a concurrent schedule procedure. Twelve pigeons were run on a series of concurrent variable-interval reinforcement schedules, from which baseline matching functions were obtained. An experimental phase followed, in which a second response key was available in one concurrent schedule alternative. For half the birds, the second key was programmed identically with the first; for the other half, the added key was programmed for extinction, with position irrelevant. Comparison of baseline and experimental matching functions revealed no systematic changes in either slope or intercept for birds in the latter group. Systematic shifts in function intercepts in the former group indicate a response bias toward the response-constrained (single-key) schedule alternative. Although contrary to the literature of preference for choice, this finding may be interpretable through an account dealing with imposed variability of responding.