Behavioral momentum: the effects of the temporal separation of rates of reinforcement.

In Part 1 of the experiment, rats responded under a variable-interval (VI) 30-s schedule and a VI 120-s schedule, with each in effect for a block of consecutive sessions. That is, the two VI schedules were presented in successive conditions. In Part 2 the VI schedules alternated each day, and in Part 3 the schedules alternated within the session as a multiple schedule. For half of the rats in Parts 1 and 2, the VI schedule alternated every few minutes within the session with a stimulus that signaled extinction. For each part, once response rates had stabilized, resistance to change was measured by prefeeding and extinction. When the schedules were examined in successive conditions (Part 1), resistance to extinction was greater under the VI 120-s schedule of reinforcement than under the VI 30-s schedule, but no consistent differences in resistance to prefeeding were observed between the two VI schedules. When the VI schedules alternated each day (Part 2), resistance to extinction was greater under the VI 120-s schedule. However, no consistent differences in resistance to prefeeding were observed between the VI schedules without extinction in Group A, but resistance to prefeeding was greater under the VI 30-s schedule for rats with the added extinction component in Group B. When the VI schedules alternated within the session as a multiple schedule (Part 3), resistance to extinction and resistance to prefeeding were greater under the VI 30-s schedule. The data suggest that different rates of reinforcement, and their accompanying discriminative stimuli, must be compared within the same session (or at least on alternate days) to produce data consistent with the behavioral momentum model.     

Mechanisms underlying the effects of unsignaled delayed reinforcement on key pecking of pigeons under variable-interval schedules.

Three experiments were conducted to test an interpretation of the response-rate-reducing effects of unsignaled nonresetting delays to reinforcement in pigeons. According to this interpretation, rates of key pecking decrease under these conditions because key pecks alternate with hopper-observing behavior. In Experiment 1, 4 pigeons pecked a food key that raised the hopper provided that pecks on a different variable-interval-schedule key met the requirements of a variable-interval 60-s schedule. The stimuli associated with the availability of the hopper (i.e., houselight and keylight off, food key illuminated, feedback following food-key pecks) were gradually removed across phases while the dependent relation between hopper availability and variable-interval-schedule key pecks was maintained. Rates of pecking the variable-interval-schedule key decreased to low levels and rates of food-key pecks increased when variable-interval-schedule key pecks did not produce hopper-correlated stimuli. In Experiment 2, pigeons initially pecked a single key under a variable-interval 60-s schedule. Then the dependent relation between hopper presentation and key pecks was eliminated by arranging a variable-time 60-s schedule. When rates of pecking had decreased to low levels, conditions were changed so that pecks during the final 5 s of each interval changed the keylight color from green to amber. When pecking produced these hopper-correlated stimuli, pecking occurred at high rates, despite the absence of a peck-food dependency. When peck-produced changes in keylight color were uncorrelated with food, rates of pecking fell to low levels. In Experiment 3, details (obtained delays, interresponse-time distributions, eating times) of the transition from high to low response rates produced by the introduction of a 3-s unsignaled delay were tracked from session to session in 3 pigeons that had been initially trained to peck under a conventional variable-interval 60-s schedule. Decreases in response rates soon after the transition to delayed reinforcement were accompanied by decreases in eating times and alterations in interresponse-time distributions. As response rates decreased and became stable, eating times increased and their variability decreased. These findings support an interpretation of the effects of delayed reinforcement that emphasizes the importance of hopper-observing behavior.     

Response Form, Force, And Number: Effects On Concurrent-schedule Performance

Six hens responded on concurrent variable-interval (key-peck) variable-interval (door-push) schedules of reinforcement in which the second-order (fixed-ratio) requirements on the alternatives (Experiment 1) or the required door forces (Experiment 2) were varied. The key-peck and door-push response (measured as fixed-ratio completion) and time data were well described by the generalized matching law. However, the manipulations of fixed-ratio requirement and required response force differed in their effects. The manipulations of fixed-ratio size affected the response and time measures differently, producing fairly constant, multiplicative biases only in terms of response allocation. It was argued that variations in fixed-ratio size necessarily change the time allocated to that response unit, and thus changes in time bias were not necessarily a fundamental effect of changing the ratio. In contrast, the changes in response bias were a fundamental result of changes in ratio size. The response-force manipulations produced similar bias shifts in terms of response and time allocation, but they appeared to combine with relative reinforcement rate to affect choice interactively. Specifically, behavior appeared to be biased towards the least effortful (i.e., key-peck) response, but the increases in door force had a larger effect on bias when the hens were making this response infrequently (on a lean schedule). The different effects of the fixed-ratio and response-force manipulations on concurrent performance were partially accounted for by the differing times required to complete each response unit under those manipulations, but this would not account for the interaction. The interaction would be consonant with increased response effort decreasing the effective value of the associated reinforcement schedule.     

Ratio size and cocaine concentration effects on oral cocaine-reinforced behavior.

Monkeys were given a choice between cocaine solutions and water under concurrent fixed-ratio reinforcement schedules. The operant response was spout contact. Six rhesus monkeys served as subjects. The cocaine concentration was varied from 0.0125 to 0.8 mg/ml, and the fixed-ratio value was varied from 8 to 128. Cocaine maintained higher response rates than did water over a wide range of conditions. Response rate and number of cocaine deliveries per session were inverted U-shaped functions of concentration. These functions were shifted to the right as the fixed ratio was increased. The number of cocaine deliveries was more persistent as fixed-ratio value was increased when the unit dose was larger rather than smaller. Cocaine consumption was analyzed as a function of unit price (fixed-ratio value divided by cocaine concentration), and unit price accounted for between 77% and 92% of the variance in cocaine consumption for individual monkeys. The current data support the claim that a drug's reinforcing effects increase directly with dose and underscore the need to gather parametric data when examining the effects of experimental manipulations on a drug-reinforced baseline.     

Effects of unsignaled delay of reinforcement on preference and resistance to change

In Phase 1, pigeons were trained on a concurrent chain in which a 3-s unsignaled delay of reinforcement was imposed on responding in a terminal link in some conditions. Preference for that terminal link was always reduced in comparison with conditions in which there was no delay, substantially so for 3 of the 4 pigeons. In Phase 2, pigeons responded in a two-component multiple schedule. The scheduled rates of reinforcement were equal, but a 3-s unsignaled delay was imposed in one component. Resistance of responding to prefeeding and extinction was reduced in the delay component for the same 3 subjects for which the data had shown strong effects of delay on preference. Systematic observation revealed differences in response topography. In the delay component, subjects oriented more closely to the key and responses were less forceful compared with the no-delay component. Our results give further evidence that preference and resistance to change covary within subjects. However, they challenge the premise that the critical determiners of preference (i.e., terminal-link value) and resistance to change (behavioral mass) may be quantified purely in terms of stimulus—reinforcer relations.     

Effects of reinforcer rate and reinforcer quality on time allocation: Extensions of matching theory to educational settings

We examined how 3 special education students allocated their responding across two concurrently available tasks associated with unequal rates and equal versus unequal qualities of reinforcement. The students completed math problems from two alternative sets on concurrent variable-interval (VI) 30-s VI 120-s schedules of reinforcement. During the equal-quality reinforcer condition, high-quality (nickels) and low-quality items ("program money" in the school's token economy) were alternated across sessions as the reinforcer for both sets of problems. During the unequal-quality reinforcer condition, the low-quality reinforcer was used for the set of problems on the VI 30-s schedule, and the high-quality reinforcer was used for the set of problems on the VI 120-s schedule. Equal- and unequal-quality reinforcer conditions were alternated using a reversal design. Results showed that sensitivity to the features of the VI reinforcement schedules developed only after the reinforcement intervals were signaled through countdown timers. Thereafter, when reinforcer quality was equal, the time allocated to concurrent response alternatives was approximately proportional to obtained reinforcement, as predicted by the matching law. However the matching relation was disrupted when, as occurs in most natural choice situations, the quality of the reinforcers differed across the response options.     

Fixed-interval performance and self-control in children.

Operant responses of 16 children (mean age 6 years and 1 month) were reinforced according to different fixed-interval schedules (with interreinforcer intervals of 20, 30, or 40 s) in which the reinforcers were either 20-s or 40-s presentations of a cartoon. In another procedure, they received training on a self-control paradigm in which both reinforcer delay (0.5 s or 40 s) and reinforcer duration (20 s or 40 s of cartoons) varied, and subjects were offered a choice between various combinations of delay and duration. Individual differences in behavior under the self-control procedure were precisely mirrored by individual differences under the fixed-interval schedule. Children who chose the smaller immediate reinforcer on the self-control procedure (impulsive) produced short postreinforcement pauses and high response rates in the fixed-interval conditions, and both measures changed little with changes in fixed-interval value. Conversely, children who chose the larger delayed reinforcer in the self-control condition (the self-controlled subjects) exhibited lower response rates and long postreinforcement pauses, which changed systematically with changes in the interval, in their fixed-interval performances.     

Continuous versus discrete dimensions of reinforcement schedules: An integrative analysis

An approach to reinforcement-schedule contingencies is presented that accommodates continuous as well as discrete effective dimensions of responses and reinforcers. College students' wheel turning was reinforced by projected reading material according to four schedule contingencies that incorporated either a discontinuous (count) or continuous (duration) dimension of the response and the reinforcer. The contingencies arranged a 1:1 correspondence between (a) response count and consequent stimulus count, (b) response duration and stimulus count, (c) response count and stimulus duration, and (d) response duration and stimulus duration. Contingencies incorporating response count produced moderate to high rates of very short-duration responses. Contingencies incorporating response duration produced very low-rate, long-duration responding. The dimension of the reinforcer had minimal or no additional effect. We suggest that incorporating duration and other continuous dimensions into schedule contingencies may improve our understanding of both laboratory and nonlaboratory behavior.     

In search of the feedback function for variable-interval schedules

Finding a theoretically sound feedback function for variable-interval schedules remains an important unsolved problem. It is important because interval schedules model a significant feature of the world: the dependence of reinforcement on factors beyond the organism's control. The problem remains unsolved because no feedback function yet proposed satisfies all the theoretical and empirical requirements. Previous suggestions that succeed in fitting data fail theoretically because they violate a newly recognized theoretical requirement: The slope of the function must approach or equal 1.0 at the origin. A function is presented that satisfies all requirements but lacks any theoretical justification. This function and two suggested by Prelec and Herrnstein (1978) and Nevin and Baum (1980) are evaluated against several sets of data. All three fitted the data well. The success of the two theoretically incorrect functions raises an empirical puzzle: Low rates of reinforcement are coupled with response rates that seem anomalously high. It remains to be discovered what this reflects about the temporal patterning of operant behavior at low reinforcement rates. A theoretically and empirically correct function derived from basic assumptions about operant behavior also remains to be discovered.     

Resistance to change and the law of effect

Three experiments using multiple schedules of reinforcement explored the implications of resistance-to-change findings for the response-reinforcer relation described by the law of effect, using both steady-state responding and responding recorded in the first few sessions of conditions. In Experiment 1, when response-independent reinforcement was increased during a third component, response rate in Components 1 and 2 decreased. This response-rate reduction was proportionately greater in a component in which reinforcer magnitude was small (2-s access to wheat) than in the component in which it was large (6-s access to wheat). However, when reinforcer rates in the two components were varied together in Experiments 2 and 3, response-rate change was the same regardless of the magnitude of reinforcers used in the two components, so that sensitivity of response rates to reinforcer rates (Experiment 2) and of response-rate ratios to reinforcer-rate ratios (Experiment 3) was unaffected by the magnitude of the reinforcers. Therefore, the principles determining resistance to change, described by behavioral momentum theory, seem not to apply when the source of behavior change is the variation of reinforcement contingencies that maintain the behavior. The use of extinction as a manipulation to study resistance to change is questioned.