A comparison of ratio and interval reinforcement schedules with comparable interreinforcement times

Pigeons were trained to peck keys on fixed-ratio and fixed-interval schedules of food reinforcement. Both schedules produced a pattern of behavior characterized as pause and run, but the relation of pausing to time between reinforcers differed for the two schedules even when mean time between reinforcers was the same. Pausing in the fixed ratio occupied less of the time between reinforcers for shorter interreinforcer times. For two of three birds, the relation was reversed at longer interreinforcer times. As an interreinforcer time elapsed, there was an increasing tendency to return to responding for the fixed interval, but a roughly constant tendency to return to responding for the fixed-ratio schedule. In Experiment 1 these observations were made for both single-reinforcement schedules and multiple schedules of fixed-ratio and fixed-interval reinforcement. In Experiment 2 the observations were extended to a comparison of fixed-ratio versus variable-interval reinforcement schedules, where the distribution of interreinforcement times in the variable interval approximated that for the fixed ratio.     

Matching, maximizing, and the behavioral unit: concurrent reinforcement of response sequences.

Pigeons pecked two keys in a probability matching situation in which four two-peck sequences were intermittently reinforced: left-left, left-right, right-left and right-right. In Phase 1, relative reinforcement rate was varied with respect to the first response of a sequence: reinforcers were differentially assigned for left-left and left-right sequences as opposed to right-left and right-right sequences. The second response of reinforced sequences occurred equally on the left and right keys across conditions. In Phase II, relative reinforcement rate was varied for sequences that involve an alternation as opposed to those that did not. The relative outputs of the different sequences matched the relative reinforcement rates for the different sequences in both phases. Relative response rates for key pecks did not always match relative reinforcement rates. The intertrial interval separating responses was varied in both phases; increases in the intertrial interval affected the relative frequency of different sequences. The results demonstrate that response sequences acted as functional units influencing choice and thus support a structural account of choice. At the same time, the matching of relative sequence proportion and relative reinforcement rate supports a matching account.     

Reinforcement: Food Signals the Time and Location of Future Food

It has long been understood that food deliveries may act as signals of future food location, and not only as strengtheners of prefood responding as the law of effect suggests. Recent research has taken this idea further—the main effect of food deliveries, or other “reinforcers”, may be signaling rather than strengthening. The present experiment investigated the ability of food deliveries to signal food contingencies across time after food. In Phase 1, the next food delivery was always equally likely to be arranged for a left‐ or a right‐key response. Conditions were arranged such that the next food delivery was likely to occur either sooner on the left (or right) key, or sooner on the just‐productive (or not‐just‐productive) key. In Phase 2, similar contingencies were arranged, but the last‐food location was signaled by a red keylight. Preference, measured in 2‐s bins across interfood intervals, was jointly controlled by the likely time and location of the next food delivery. In Phase 1, when any food delivery signaled a likely sooner next food delivery on a particular key, postfood preference was strongly toward that key, and moved toward the other key across the interreinforcer interval. In other conditions in which food delivery on the two keys signaled different subsequent contingencies, postfood preference was less extreme, and quickly moved toward indifference. In Phase 2, in all three conditions, initial preference was strongly toward the likely‐sooner food key, and moved to the other key across the interfood interval. In both phases, at a more extended level of analysis, sequences of same‐key food deliveries caused a small increase in preference for the just‐productive key, suggesting the presence of a “reinforcement effect”, albeit one that was very small.     

Effects of Rate of Reinforcement and Rate of Change on Choice Behaviour in Transition

In two experiments with pigeons, a single variable-interval schedule assigned reinforcers to two response keys on a percentage basis. The percentage of reinforcers assigned to each key was changed every few sessions, and subjects' choice responses were recorded before and after each change. In Experiment 1, the overall rate of reinforcement was varied across conditions. The pigeons' choice responses adapted more quickly to a change in the reinforcement percentages when the overall reinforcement rates were higher, but acquisition rates varied by only about a factor of 3, whereas reinforcement rates were varied by about a factor of 9. In Experiment 2, the reinforcement percentages changed about every 8 sessions in Phases 1 and 3, but every 1 or 2 sessions in Phase 2. Pigeons' choice responses adapted to a change in reinforcement percentages more quickly in Phase 2 than in Phases 1 and 3. The results from both experiments pose difficulties for several prominent models of transitional choice behaviour. The results suggest that each successive reinforcer has more impact on a subject's subsequent choice behaviour when the overall rate of reinforcement is lower and when the reinforcement contingencies have changed frequently in the recent past.     

Resistance to extinction following variable-interval reinforcement: reinforcer rate and amount

Rats obtained food-pellet reinforcers by nose poking a lighted key. Experiment 1 examined resistance to extinction following single-schedule training with different variable-interval schedules, ranging from a mean interval of 16 min to 0.25 min. That is, for each schedule, the rats received 20 consecutive daily baseline sessions and then a session of extinction (i.e., no reinforcers). Resistance to extinction (decline in response rate relative to baseline) was negatively related to the rate of reinforcers obtained during baseline, a relation analogous to the partial-reinforcement-extinction effect. A positive relation between these variables emerged, however, when the unit of extinction was taken as the mean interreinforcer interval that had been in effect during training (i.e., as an omitted reinforcer during extinction). In a second experiment, rats received blocks of training sessions, all with the same variable-interval schedule but with a reinforcer of four pellets for some blocks and one pellet for others. Resistance to extinction was greater following training with the larger (four pellets) than with the smaller (one pellet) reinforcer. Taken together, these results support the principle that greater reinforcement during training (e.g., higher rate or larger amount) engenders greater resistance to extinction even when the different conditions of reinforcement are varied between blocks of sessions.     

Independence of terminal-link entry rate and immediacy in concurrent chains

In Phase 1, 4 pigeons were trained on a three-component multiple concurrent-chains procedure in which components differed only in terms of relative terminal-link entry rate. The terminal links were variable-interval schedules and were varied across four conditions to produce immediacy ratios of 4:1, 1:4, 2:1, and 1:2. Relative terminal-link entry rate and relative immediacy had additive and independent effects on initial-link response allocation, and the data were well-described by a generalized-matching model. Regression analyses showed that allowing sensitivity to immediacy to vary across components produced only trivial increases in variance accounted for. Phase 2 used a three-component concurrent-schedules procedure in which the schedules were the same as the initial links of Phase 1. Across two conditions, the relative reinforcer magnitude was varied. Sensitivity to relative reinforcer rate was independent of relative magnitude, confirming results of prior studies. Sensitivity to relative reinforcer rate in Phase 2 did not vary systematically across subjects compared to sensitivity to relative entry rate in Phase 1, and regression analyses confirmed again that only small increases in variance accounted for were obtained when sensitivities were estimated independently compared with a single estimate for both phases. Overall, the data suggest that conditioned and primary reinforcers have functionally equivalent effects on choice and support the independence of relative terminal-link entry rate and immediacy as determiners of response allocation. These results are consistent with current models for concurrent chains, including Grace's (1994) contextual choice model and Mazur's (2001) hyperbolic value-added model.     

EFFECTS OF POINT‐LOSS PUNISHERS ON HUMAN SIGNAL‐DETECTION PERFORMANCE

Three experiments using human participants varied the distribution of point‐gain reinforcers or point‐loss punishers in two‐alternative signal‐detection procedures. Experiment 1 varied the distribution of point‐gain reinforcers for correct responses (Group A) and point‐loss punishers for errors (Group B) across conditions. Response bias varied systematically as a function of the relative reinforcer or punisher frequencies. Experiment 2 arranged two conditions — one where an unequal ratio of reinforcement (5:1 or 1:5) was presented without punishment (R‐only), and another where the same reinforcer ratio was presented with an equal distribution of point‐loss punishers (R+P). Response bias was significantly greater in the R‐only condition than the R+P condition, supporting a subtractive model of punishment. Experiment 3 varied the distribution of point‐gain reinforcers for correct responses across four unequal reinforcer ratios (5:1, 2:1, 1:2, 1:5) both without (R‐only) and with (R+P) an equal distribution of point‐loss punishers for errors. Response bias varied systematically with changes in relative reinforcer frequency for both R‐only and R+P conditions, with 5 out of 8 participants showing increases in sensitivity estimates from R‐only to R+P conditions. Overall, the results indicated that punishers have similar but opposite effects to reinforcers in detection procedures and that combined reinforcer and punisher effects might be better modeled by a subtractive punishment model than an additive punishment model, consistent with research using concurrent‐schedule choice procedures.     

Choice in a variable environment: every reinforcer counts

Six pigeons were trained in sessions composed of seven components, each arranged with a different concurrent-schedule reinforcer ratio. These components occurred in an irregular order with equal frequency, separated by 10-s blackouts. No signals differentiated the different reinforcer ratios. Conditions lasted 50 sessions, and data were collected from the last 35 sessions. In Part 1, the arranged overall reinforcer rate was 2.22 reinforcers per minute. Over conditions, number of reinforcers per component was varied from 4 to 12. In Part 2, the overall reinforcer rate was six per minute, with both 4 and 12 reinforcers per component. Within components, log response-allocation ratios adjusted rapidly as more reinforcers were delivered in the component, and the slope of the choice relation (sensitivity) leveled off at moderately high levels after only about eight reinforcers. When the carryover from previous components was taken into account, the number of reinforcers in the components appeared to have no systematic effect on the speed at which behavior changed after a component started. Consequently, sensitivity values at each reinforcer delivery were superimposable. However, adjustment to changing reinforcer ratios was faster, and reached greater sensitivity values, when overall reinforcer rate was higher. Within a component, each successive reinforcer from the same alternative ("confirming") had a smaller effect than the one before, but single reinforcers from the other alternative ("disconfirming") always had a large effect. Choice in the prior component carried over into the next component, and its effects could be discerned even after five or six reinforcement and nonreinforcement is suggested.     

Sensitivity of time allocation to an overall reinforcer rate feedback function in concurrent interval schedules

Six pigeons were trained on concurrent variable-interval schedules in which feedback functions arranged that the overall reinforcer rate either (a) was independent of preference, (b) decreased with increasing absolute preference, or (c) increased with increasing absolute preference. In Experiment 1, the reinforcer rate in an interreinforcement interval was determined by the absolute time-allocation ratio in the previous interval. When arranged reinforcer ratios were varied, there was no evidence of control over preference by overall reinforcer rate. In Experiment 2, the feedback function arranged that reinforcer rates were an inverse function of absolute preference, and window durations were fixed times. In Phase 1, using schedules that provided a four-to-one reinforcer ratio, the window duration was decreased from 20 s to 5 s over four conditions. Then, in Phases 2 and 3, the arranged reinforcer ratios were varied. In Phase 2, the reinforcer rate in the current 5-s time window was determined by preference in the previous 5-s window, and in Phase 3, the window durations were 20 s. Again, there was no indication of control by obtained overall reinforcer rate. These data call into question theories that suggest that the process underlying matching is one of maximizing overall reinforcer rates, or that preference in concurrent aperiodic schedules is controlled to any extent by overall reinforcer rate. They also question the notion that concurrent-schedule preference is controlled by molecular maximizing.     

Response–reinforcer dependency and resistance to change

The effects of the response–reinforcer dependency on resistance to change were studied in three experiments with rats. In Experiment 1, lever pressing produced reinforcers at similar rates after variable interreinforcer intervals in each component of a two‐component multiple schedule. Across conditions, in the fixed component, all reinforcers were response‐dependent; in the alternative component, the percentage of response‐dependent reinforcers was 100, 50 (i.e., 50% response‐dependent and 50% response‐independent) or 10% (i.e., 10% response‐dependent and 90% response‐independent). Resistance to extinction was greater in the alternative than in the fixed component when the dependency in the former was 10%, but was similar between components when this dependency was 100 or 50%. In Experiment 2, a three‐component multiple schedule was used. The dependency was 100% in one component and 10% in the other two. The 10% components differed on how reinforcers were programmed. In one component, as in Experiment 1, a reinforcer had to be collected before the scheduling of other response‐dependent or independent reinforcers. In the other component, response‐dependent and ‐independent reinforcers were programmed by superimposing a variable‐time schedule on an independent variable‐interval schedule. Regardless of the procedure used to program the dependency, resistance to extinction was greater in the 10% components than in the 100% component. These results were replicated in Experiment 3 in which, instead of extinction, VT schedules replaced the baseline schedules in each multiple‐schedule component during the test. We argue that the relative change in dependency from Baseline to Test, which is greater when baseline dependencies are high rather than low, could account for the differential resistance to change in the present experiments. The inconsistencies in results across the present and previous experiments suggest that the effects of dependency on resistance to change are not well understood. Additional systematic analyses are important to further understand the effects of the response–reinforcer relation on resistance to change and to the development of a more comprehensive theory of behavioral persistence.     

Fixed-interval performance and self-control in children.

Operant responses of 16 children (mean age 6 years and 1 month) were reinforced according to different fixed-interval schedules (with interreinforcer intervals of 20, 30, or 40 s) in which the reinforcers were either 20-s or 40-s presentations of a cartoon. In another procedure, they received training on a self-control paradigm in which both reinforcer delay (0.5 s or 40 s) and reinforcer duration (20 s or 40 s of cartoons) varied, and subjects were offered a choice between various combinations of delay and duration. Individual differences in behavior under the self-control procedure were precisely mirrored by individual differences under the fixed-interval schedule. Children who chose the smaller immediate reinforcer on the self-control procedure (impulsive) produced short postreinforcement pauses and high response rates in the fixed-interval conditions, and both measures changed little with changes in fixed-interval value. Conversely, children who chose the larger delayed reinforcer in the self-control condition (the self-controlled subjects) exhibited lower response rates and long postreinforcement pauses, which changed systematically with changes in the interval, in their fixed-interval performances.     

Choice in transition: Replication and extension to preschool children in a naturalistic setting

This study replicated previous basic research into the dynamics of choice and extended this analysis to children's behavior in a naturalistic setting. Two preschoolers with disabilities were observed interacting with their teachers at baseline and during an experimental analysis involving four pairs of concurrent variable‐interval schedules of adult attention implemented by an experimenter. Each child was exposed to four experimental phases in which the relative reinforcer rates for on‐ and off‐task behavior were 10:1, 1:1, 1:10, and reversed back to 10:1. The 10:1 phase was designed to mimic the same schedules and types of adult attention observed at baseline. We used the generalized matching equation to model steady‐state behavior at the end of the transition phases and to evaluate changes in sensitivity at various points throughout the phases. Choice in transition was evaluated by plotting log behavior ratios by session, cumulated time on‐ and off‐task and cumulated attention for on‐ and off‐task behavior by session, and interreinforcer behavior ratios following different sequences of the first four reinforcer deliveries. The generalized matching equation accounted for a large proportion of variance in steady‐state responding, sensitivity values increased steadily throughout the phases, patterns of choice in transition were similar to those reported in basic research, and interreinforcer preference generally shifted toward the just‐reinforced alternative. These findings are consistent with previous basic research and support the generality of the dynamics of choice to children's on‐ and off‐task behavior reinforced by adult attention.     

The behavioral theory of timing: Reinforcer rate determines pacemaker rate

In the behavioral theory of timing, pulses from a hypothetical Poisson pacemaker produce transitions between states that are correlated with adjunctive behavior. The adjunctive behavior serves as a discriminative stimulus for temporal discriminations. The present experiments tested the assumption that the average interpulse time of the pacemaker is proportional to interreinforcer interval. Responses on a left key were reinforced at variable intervals for the first 25 s since the beginning of a 50-s trial, and right-key responses were reinforced at variable intervals during the second 25 s. Psychometric functions relating proportion of right-key responses to time since trial onset, in 5-s intervals across the 50-s trial, were sigmoidal in form. Average interpulse times derived by fitting quantitative predictions from the behavioral theory of timing to obtained psychometric functions decreased when the interreinforcer interval was decreased and increased when the interreinforcer interval was increased, as predicted by the theory. In a second experiment, average interpulse times estimated from trials without reinforcement followed global changes in interreinforcer interval, as predicted by the theory. Changes in temporal discrimination as a function of interreinforcer interval were therefore not influenced by the discrimination of reinforcer occurrence. The present data support the assumption of the behavioral theory of timing that interpulse time is determined by interreinforcer interval.     

Concurrent-schedule performance: Effects of relative and overall reinforcer rate

Six pigeons were trained to respond on two keys, each of which provided reinforcers on an arithmetic variable-interval schedule. These concurrent schedules ran nonindependently with a 2-s changeover delay. Six sets of conditions were conducted. Within each set of conditions the ratio of reinforcers available on the two alternatives was varied, but the arranged overall reinforcer rate remained constant. Each set of conditions used a different overall reinforcer rate, ranging from 0.22 reinforcers per minute to 10 reinforcers per minute. The generalized matching law fit the data from each set of conditions, but sensitivity to reinforcer frequency (a) decreased as the overall reinforcer rate decreased for both time allocation and response allocation based analyses of the data. Overall response rates did not vary with changes in relative reinforcer rate, but decreased with decreases in overall reinforcer rate. Changeover rates varied as a function of both relative and overall reinforcer rates. However, as explanations based on changeover rate seem unable to deal with the changes in generalized matching sensitivity, discrimination accounts of choice may offer a more promising interpretation.     

Do common elements predict class merger: A test of Sidman's theory of equivalence

The study presented here investigated the effect of common and uncommon elements on class merger as predicted by Sidman in his reconceptualization of stimulus equivalence suggesting that common elements among contingencies can facilitate emergent performances (1994, 1997, 2000). Eight adult participants were exposed to a procedure that arranged for stimulus–reinforcer correlations in Phase 1 and response–reinforcer correlations in Phase 2 of a 3-phase study. In the common element group, the visual images serving as reinforcers were the same in Phase 1 and Phase 2. In the uncommon elements group, the images serving as reinforcers were different in Phases 1 and 2. In Phase 3, participants were given an opportunity to respond but no feedback was programmed. The results showed that participants' responding was well differentiated in the common element group and undifferentiated in the uncommon elements group. These results are predicted by Sidman's revised formulation of the provenance and scope of equivalence relations. Specifically, these data support Sidman's (1994, 1997, 2000) suggestion that elements of a contingency enter into an equivalence class and common elements among contingencies are sufficient to produce class mergers. The findings highlight an emergent simple discrimination and raise some interesting considerations about the definition of equivalence under the new formulation.     

Contrast and undermatching as a function of reinforcer duration and quality during multiple schedules

Eight pigeons pecked keys under multiple variable-interval two-minute variable-interval two-minute schedules. In Experiment 1, the reinforcers were 2, 4, or 8 seconds access to a food magazine. In Experiments 2 and 3, the reinforcers were grains that had been determined to be most-, moderately-, or non-preferred. Both positive and negative behavioral contrast occurred when the reinforcers in one component were held constant and the duration or type of reinforcer obtained in the other component varied. Undermatching occurred when the relative rate of responding during a component was plotted as a function of the relative duration of the reinforcers in that component.     

NONCONTINGENT REINFORCEMENT AS TREATMENT FOR SEVERE PROBLEM BEHAVIOR: SOME PROCEDURAL VARIATIONS

Noncontingent reinforcement (NCR) as a treatment for problem behavior has typically included (a) continuous access to reinforcers at the onset of treatment and (b) extinction. We extended research on NCR by conducting a three-phase preliminary investigation of these components. In Phase 1, a functional analysis showed that the problem behavior of 3 participants with developmental disabilities was maintained by tangible positive reinforcement. In Phase 2, treatment started with the initial NCR interval based on the latency to the first problem behavior during baseline. In Phase 3, treatment consisted of NCR without extinction to determine whether extinction was an essential treatment component. Results showed that the initial NCR schedule based on latency (Phase 2) and NCR without extinction (Phase 3) were effective for reducing rates of problem behavior compared with baseline. Findings are discussed regarding the initial schedule of reinforcement and extinction as components of NCR.     

Sensitivity to reinforcer duration in a self-control procedure

In a concurrent-chains procedure, pigeons' responses on left and right keys were followed by reinforcers of different durations at different delays following the choice responses. Three pairs of reinforcer delays were arranged in each session, and reinforcer durations were varied over conditions. In Experiment 1 reinforcer delays were unequal, and in Experiment 2 reinforcer delays were equal. In Experiment 1 preference reversal was demonstrated in that an immediate short reinforcer was chosen more frequently than a longer reinforcer delayed 6 s from the choice, whereas the longer reinforcer was chosen more frequently when delays to both reinforcers were lengthened. In both experiments, choice responding was more sensitive to variations in reinforcer duration at overall longer reinforcer delays than at overall shorter reinforcer delays, independently of whether fixed-interval or variable-interval schedules were arranged in the choice phase. We concluded that preference reversal results from a change in sensitivity of choice responding to ratios of reinforcer duration as the delays to both reinforcers are lengthened.     

The interaction between stimulus and reinforcer control on remembering.

In a symbolic matching-to-sample task, 6 pigeons obtained food by pecking a red side key when the brighter of two white lights had been presented on the center key and by pecking a green side key when the dimmer of two white lights had been presented on the center key. Across Part 1 and Parts 6 to 10, the delay between sample-stimulus presentation and the availability of the choice keys was varied between 0 s and 25 s. Across Parts 1 to 5, the delay between the emission of a correct choice and the delivery of a reinforcer was varied between 0 s and 30 s. Although increasing both types of delay decreased stimulus discriminability, lengthening the stimulus-choice delay produced a greater decrement in choice accuracy than did lengthening the choice-reinforcer delay. Additionally, the relative reinforcer rate for correct choice was varied across both types of delay. The sensitivity of behavior to the distribution of reinforcers decreased as discriminability decreased under both procedures. These data are consistent with the view, based on the generalized matching law, that sample stimuli and reinforcers interact in their control over remembering.     

Divided stimulus control: Which key did you peck,or what color was it?

Responding on concurrent schedules produced a conditional discrimination (Phases 1 and 2), asking either which peck produced the event, or which color the keys were when the event was produced. In Phases 3 and 4, reinforcer delivery or a delay in blackout was interpolated between responding and the conditional discrimination. In Phase 1, location versus color discrimination accuracy was controlled by the relative reinforcer frequency for correct responses to these questions (divided stimulus control). In Phases 2 to 4, relative reinforcer frequency for correct responses to these questions was .5, and the relative frequency with which concurrent‐schedule responses produced the questions was varied. This variation had no clear effect on the accuracy of reporting Location or Color. These results are consistent with the model of divided control suggested by Davison and Elliffe (2010). Arranging a 3‐s reinforcer between responding and choice decreased both color and location accuracy, but a 3‐s delay only decreased location accuracy. Thus, in concurrent‐schedule performance, both ambient stimuli prior to a reinforcer and the location of the just‐reinforced response are available as discriminative stimuli following the reinforcer. Control of postreinforcer responding is divided between these according to their association with the relative frequency of subsequent reinforcers.