Soft commitment: self-control achieved by response persistence.

With reinforcement contingent on a single peck on either of two available keys (concurrent continuous reinforcement schedules) 4 pigeons, at 80% of free-feeding weights, preferred a smaller-sooner reinforcer (2.5 s of mixed grain preceded by a 0.5-s delay) to a larger-later reinforcer (4.5 s of mixed grain preceded by a 3.5-s delay). However, when the smaller-sooner and larger-later reinforcers were contingent on a concurrent fixed-ratio 31 schedule (the first 30 pecks distributed in any way on the two keys), all pigeons obtained the larger-later reinforcer much more often than they did when only a single peck was required. This "self-control" was achieved by beginning to peck the key leading to the larger-later reinforcer and persisting on that key until reinforcement occurred. We call this persistence "soft commitment" to distinguish it from strict commitment, in which self-control is achieved by preventing changeovers. Soft commitment also effectively achieved self-control when a brief (1-s) signal was inserted between the 30th and 31st response of the ratio and with concurrent fixed-interval 30-s schedules (rather than ratio schedules) of reinforcement. In a second experiment with the same subjects, the fixed ratio was interrupted by darkening both keys and lighting a third (center) key on which pecking was required for various fractions of the fixed-ratio count. The interruption significantly reduced self-control. When interruption was complete (30 responses on the center key followed by a single choice response), pigeons chose the smaller-sooner reinforcer as frequently as they did when only a single choice response was required.     

Quasi-reinforcement: control of responding by a percentage-reinforcement schedule

When a variable-interval schedule of reinforcement was segmented into small fixed-interval components, with reinforcements following some components and brief blackouts following the others, rate of responding doubled and a positively accelerated pattern within each component was obtained. Presented according to this percentage reinforcement paradigm, the blackouts approximated the functions of a food reinforcer. These effects occurred only when the behavior sequence required to produce reinforcement was identical to that required to produce blackout. The quasi-reinforcing effects of these blackout stimuli suggest that a neutral stimulus need not occasion or accompany a primary reinforcer to acquire reinforcing properties.     

Food priming: A therapeutic tool to increase the percentage of meals bought by chronic mental patient

In a motivating ward environment for schizophrenics, three methods of maximizing food as an effective reinforcer were tested. On the basis of pre-treatment measures, patients were classified as (1) “non-chronic meal missers” who served as control patients (Group A), (2) “chronic meal missers” because they had no tokens (Group B), or (3) “chronic meal missers” because they chose not to eat but had tokens (Group C). One of the following three experimental treatments was then administered when a patient missed a meal : (1) visual food priming (observing others eat), (2) oral food priming (sampling one teaspoon of each type of food for that meal), and (3) free meal (being offered a free meal when it was missed). Oral and visual food priming were found to be equally effective in increasing the percentage of meals bought by patients. A post-treatment follow-up for Group B found that this increase was even greater during the follow-up phase. The percentage of meals bought by patients in the free meal condition of Group B decreased during the treatment phase but was significantly above pre-treatment measures during the follow-up phase, suggesting that complete reinforcer sampling may also be an effective technique for increasing the percentage of meals bought.     

Reinforcement and instructions with mental patients

An attempt was made to modify a socially desirable response of mental patients. It was found that instructions to the patients had no enduring effect unless accompanied by reinforcement. Also, it was found that reinforcement was not effective unless the reinforcement procedure was accompanied by instructions that specified the basis for the reinforcement. Maximum change in behavior was obtained when the reinforcement procedure took advantage of the existing verbal repertoire of the patients. A significant methodological finding was that substantial modification of the behavior of psychotics could be achieved by briefly delaying, rather than withholding, food reinforcement.     

Concurrent schedules: short- and long-term effects of reinforcers

Five pigeons were trained on concurrent variable-interval schedules in a switching-key procedure. The overall rate of reinforcement was constant in all conditions, and the ratios of reinforcers obtainable on the two alternatives were varied over seven levels. Each condition remained in effect for 65 sessions, and the last 50 sessions of data from each condition were analyzed. The most recently obtained reinforcer had the largest effect on current preference, but each of the eight previously obtained reinforcers had a small measurable effect. These effects were larger when the reinforcer ratio was more extreme. A longer term effect of reinforcement was also evident, which changed as a function of the reinforcer ratio arranged. More local analyses showed regularities at a reinforcer-by-reinforcer level and large transient movements in preference toward the just-reinforced alternative immediately following reinforcers, followed by a return to stable levels that were related to the reinforcer ratio in effect. The present data suggest that the variables that control choice have both short- and long-term effects and that the short-term effects increased when the reinforcer ratios arranged were more extreme.     

A comparison of delays and ratio requirements in self-control choice.

In a discrete-trial procedure, pigeons could choose between 2-s and 6-s access to grain by making a single key peck. In Phase 1, the pigeons obtained both reinforcers by responding on fixed-ratio schedules. In Phase 2, they received both reinforcers after simple delays, arranged by fixed-time schedules, during which no responses were required. In Phase 3, the 2-s reinforcer was available through a fixed-time schedule and the 6-s reinforcer was available through a fixed-ratio schedule. In all conditions, the size of the delay or ratio leading to the 6-s reinforcer was systematically increased or decreased several times each session, permitting estimation of an "indifference point," the schedule size at which a subject chose each alternative equally often. By varying the size of the schedule for the 2-s reinforcer across conditions, several such indifference points were obtained from both fixed-time conditions and fixed-ratio conditions. The resulting "indifference curves" from fixed-time conditions and from fixed-ratio conditions were similar in shape, and they suggested that a hyperbolic equation describes the relation between ratio size and reinforcement value as well as the relation between reinforcer delay and its reinforcement value. The results from Phase 3 showed that subjects chose fixed-time schedules over fixed-ratio schedules that generated the same average times between a choice response and reinforcement.     

Substitutability between conditioned and primary reinforcers in discrimination acquisition.

Rats and pigeons were trained on a series of reversals of a conditional simultaneous discrimination. The percentage of reinforcement for correct trials was varied across reversals. When nonreinforced correct trials produced the same feedback as incorrect trials, the number of errors to reach an acquisition criterion was greater for smaller percentages of reinforcement, but the number of reinforcers required was either approximately constant or smaller for the smaller percentages. When a stimulus paired with food (the conditioned reinforcer) was added on nonreinforced correct trials, both measures were substantially decreased. When the same stimulus was presented, but without a history of food pairing, learning rate was similar to when no stimulus was presented on nonreinforced trials. The results provide direct evidence that conditioned reinforcers may substitute, although imperfectly, for a primary reinforcer, and that pairing with the primary reinforcer is a necessary condition for such substitutability to occur.     

Reinforcing self-help group activities of welfare recipients

The attendance rates of welfare recipients at self-help group meetings was compared when supplementary reinforcement was given for attending and when it was not. Reinforcers included donated items, such as clothing and household goods, and information about welfare services. It was found that the attendance at meetings averaged three recipients per meeting without reinforcement and 15 recipients per meeting with reinforcement. There was evidence that attendance was associated with participation in other self-help activities. It was concluded that practical forms of reinforcement can be found to maintain the participation of severely disadvantaged populations in self-help groups.     

The effects of the stimulus-reinforcer correlation in a discrete-trials IRT>t procedure

The correlation between a keylight and food in a discrete-trials, interresponse-time-greater-than 6-sec (IRT>6-sec) procedure was varied by manipulating the rate of response-independent food presentation in the intertrial interval. When the correlation was positive, the rates of pecking in the IRT>6-sec condition were high and food was obtained on only about 5% of the trials. Likewise, responding was maintained at a high rate in yoked birds that received the same presentations of the light and food as the birds in the IRT>6-sec condition. When the rate of reinforcement between trials was equated to or made greater than the rate of reinforcement within trials, the response rate decreased for all birds, and those decreases were considerably larger for the yoked birds. However, the percentage of trials in which reinforced responses occurred under the IRT>6-sec procedure did not increase substantially when the light and food were either uncorrelated or negatively correlated. The percentage of trials in which a reinforcer was obtained increased when the keylight was left on continuously and the discriminative stimulus was not presented on the key. The results show that the stimulus-reinforcer correlation affects responding in the discrete-trials IRT>6-sec procedure, but that the effects of the stimulus-reinforcer correlation vary as a function of whether reinforcement is response-dependent or response-independent. The differences between the effects of response-independent and response-dependent pairings and nonpairings of the light and food are best accounted for in terms of differences in the control of responding by background stimuli.     

The measurement and reinforcement of behavior of psychotics

An attempt was made to strengthen behaviors of psychotics by applying operant reinforcement principles in a mental hospital ward. The behaviors studied were necessary and/or useful for the patient to function in the hospital environment. Reinforcement consisted of the opportunity to engage in activities that had a high level of occurrence when freely allowed. Tokens were used as conditioned reinforcers to bridge the delay between behavior and reinforcement. Emphasis was placed on objective definition and quantification of the responses and reinforcers and upon programming and recording procedures. Standardizing the objective criteria permitted ward attendants to administer the program. The procedures were found to be effective in maintaining the desired adaptive behaviors for as long as the procedures were in effect. In a series of six experiments, reinforced behaviors were considerably reduced when the reinforcement procedure was discontinued; the adaptive behaviors increased immediately when the reinforcement procedure was re-introduced.     

Behavioral contrast as a function of the temporal location of reinforcement

Pigeons were trained on a multiple variable-interval variable-interval schedule of reinforcement. One component was then changed to a variation of a fixed-interval schedule in which the same rate of reinforcement was obtained as previously but the location of the reinforcer was fixed within the component. The effects of different temporal locations were compared. An increase in response rate for the unchanged variable-interval component (behavioral contrast) occurred when the reinforcer was located in the middle or at the end of the FI component, but response suppression occurred when it was located at the beginning of the component. The pattern of results cannot be explained by any previous theories of contrast. The overall response rates, and the pattern of local rates within the components, were consistent with the hypothesis that the major determinant of the contrast effect was the transition to a lower reinforcement rate following the unchanged component.     

Effects of reinforcer rate and reinforcer quality on time allocation: Extensions of matching theory to educational settings

We examined how 3 special education students allocated their responding across two concurrently available tasks associated with unequal rates and equal versus unequal qualities of reinforcement. The students completed math problems from two alternative sets on concurrent variable-interval (VI) 30-s VI 120-s schedules of reinforcement. During the equal-quality reinforcer condition, high-quality (nickels) and low-quality items ("program money" in the school's token economy) were alternated across sessions as the reinforcer for both sets of problems. During the unequal-quality reinforcer condition, the low-quality reinforcer was used for the set of problems on the VI 30-s schedule, and the high-quality reinforcer was used for the set of problems on the VI 120-s schedule. Equal- and unequal-quality reinforcer conditions were alternated using a reversal design. Results showed that sensitivity to the features of the VI reinforcement schedules developed only after the reinforcement intervals were signaled through countdown timers. Thereafter, when reinforcer quality was equal, the time allocated to concurrent response alternatives was approximately proportional to obtained reinforcement, as predicted by the matching law. However the matching relation was disrupted when, as occurs in most natural choice situations, the quality of the reinforcers differed across the response options.     

Effects of methylphenidate and morphine on delay-discount functions obtained within sessions

Four rats responded under a "self-control" procedure designed to obtain delay-discount functions within sessions. Each session consisted of seven blocks, with seven trials within each block. Each block consisted of two initial forced-choice trials followed by five free-choice trials. On choice trials, the rats could press either of two retractable levers. A press on one lever was followed by presentation of a smaller reinforcer (a single dipper presentation of a sucrose solution); a press on the other lever was followed by presentation of a larger reinforcer (four consecutive dipper presentations). The delay associated with the smaller reinforcer always was 0 s, whereas the signaled delay associated with the larger reinforcer increased across blocks (from 0 to 50 s). Under these conditions, the percentage of choices of the larger reinforcer decreased across blocks, and relatively reliable delay-discount functions were obtained within sessions. Doses of methylphenidate (1.0 to 17.0 mg/kg) and morphine (0.3 to 17.0 mg/kg) were then administered prior to selected sessions. Typically, intermediate doses of methylphenidate shifted the discount functions to the right (increased choices of the larger reinforcer). For 2 of the rats, this effect was pronounced; for the other 2 rats, this effect occurred after the range of delays for the larger reinforcer was decreased (0 to 20 s). On the other hand, in most cases morphine produced a slight leftward shift in the discount function (decreased choices of the larger reinforcer). The present procedure appears to be a useful and efficient method to characterize drug effects on an entire delay-discount function. As with many procedures used to study self-control choices, however, sources of control other than reinforcement delay and amount may have been operating in the present study, and these sources must be considered when interpreting drug effects.     

Resurgence Following Higher or Lower Quality Alternative Reinforcement

Resurgence is a temporary increase in a previously suppressed target behavior following a worsening in reinforcement conditions. Previous studies have examined how higher rates or magnitudes of alternative reinforcement affect suppression of the target behavior and subsequent resurgence. However, there has been no investigation of the effects of higher versus lower qualities of alternative reinforcement on resurgence. Using a three-phase resurgence preparation with rats, the present experiments examined the effects of an alternative reinforcer that was of higher (Experiment 1) or lower (Experiment 2) quality than the reinforcer that had previously maintained the target behavior. The results of both experiments showed greater reductions in target behavior with a higher quality alternative reinforcer and larger increases in target responding when a higher quality alternative reinforcer was removed. Along with prior findings with higher rates and magnitudes of alternative reinforcement, these findings suggest that variations in reinforcer dimensions that increase the efficacy of alternative reinforcement also tend to increase resurgence when alternative reinforcement is removed. The results are discussed in terms of the resurgence as choice in context model and in terms of potential clinical implications.     

Choice with delayed and probabilistic reinforcers: effects of variability, time between trials, and conditioned reinforcers.

In a discrete-trials procedure with pigeons, a response on a green key led to a 4-s delay (during which green houselights were lit) and then a reinforcer might or might not be delivered. A response on a red key led to a delay of adjustable duration (during which red houselights were lit) and then a certain reinforcer. The delay was adjusted so as to estimate an indifference point--a duration for which the two alternatives were equally preferred. Once the green key was chosen, a subject had to continue to respond on the green key until a reinforcer was delivered. Each response on the green key, plus the 4-s delay that followed every response, was called one "link" of the green-key schedule. Subjects showed much greater preference for the green key when the number of links before reinforcement was variable (averaging four) than when it was fixed (always exactly four). These findings are consistent with the view that probabilistic reinforcers are analogous to reinforcers delivered after variable delays. When successive links were separated by 4-s or 8-s "interlink intervals" with white houselights, preference for the probabilistic alternative decreased somewhat for 2 subjects but was unaffected for the other 2 subjects. When the interlink intervals had the same green houselights that were present during the 4-s delays, preference for the green key decreased substantially for all subjects. These results provided mixed support for the view that preference for a probabilistic reinforcer is inversely related to the duration of conditioned reinforcers that precede the delivery of food.     

Effects of thinning the rate at which the alternative behavior is reinforced on resurgence of an extinguished instrumental response

Three experiments with rats examined the effects of thinning the rate of reinforcement for the alternative behavior in the resurgence paradigm. In all experiments, pressing one lever (L1) was first reinforced and then extinguished while pressing a second alternative lever (L2) was then reinforced. When L2 responding was then extinguished, L1 responses "resurged." Resurgence was always observed when L2 was reinforced on an unchanging reinforcement schedule during Phase 2. However, other rats received systematic decreases in the rate of L2 reinforcement before extinction of L2 began. Such a "thinning" procedure was predicted to reduce final resurgence by associating L1 extinction with longer and longer periods without a reinforcer. The procedure did reduce the resurgence effect observed when L2 was put on extinction (Experiment 3). However, in each experiment, thinned groups also returned to L1 responding, and continued to make L1 responses, while the reinforcement schedule for L2 was being thinned. Fine-grained analysis of behavior in time suggested that this early resurgence was not due to adventitious reinforcement of L1, occasion setting of L1 by reinforcer presentation, or the entrainment of L1 as a schedule-induced interim behavior. The results are overall consistent with the hypothesis that resurgence is a renewal effect in which extinguished L1 responding recovers when the context provided by the L2 reinforcement schedule is changed. Challenges for this view are also discussed.     

Influences of delay and rate of reinforcement on discrete-trial choice

An adjusting procedure was used to measure pigeons' preferences among alternatives that differed in the duration of a delay before reinforcement and of an intertrial interval (ITI) after reinforcement. In most conditions, a peck at a red key led to a fixed delay, followed by reinforcement, a fixed ITI, and then the beginning of the next trial. A peck at a green key led to an adjustable delay, reinforcement, and then the next trial began without an ITI. The purpose of the adjusting delay was to estimate an indifference point, or a delay that made a subject approximately indifferent between the two alternatives. As the ITI for the red key increased from 0 s to 60 s, the green-key delay at the indifference point increased systematically but only slightly. The fact that there was some increase showed that pigeons' choices were controlled by more than simply the delay to the next reinforcer. One interpretation of these results is that besides delay of reinforcement, rate of reinforcement also influenced choice. However, an analysis that ignored reinforcement rate, but considered the delays between a choice response and the reinforcers on subsequent trials, was able to account for most of the obtained increases in green-key delays. It was concluded that in this type of discrete-trial situation, rate of reinforcement exerts little control over choice behavior, and perhaps none at all.     

Human Sensitivity To Concurrent Schedules Of Reinforcement: Effects Of Observing Schedule-correlated Stimuli

The determinants of human sensitivity to concurrent variable-interval variable-interval schedules of reinforcement have been difficult to identify, in part because of procedural differences separating published experiments. This experiment investigated vigilance to stimuli correlated with concurrent schedules. Across phases, 3 college students were provided with either no schedule-correlated stimuli, an observing response that provided brief access to the stimuli, or a contingency that required the subject to identify the stimulus correlated with the source of each obtained reinforcer. Sensitivity, as quantified by the generalized matching equation, was low when no stimuli were available. When the stimuli were response contingent, 1 subject observed them, and her behavior became more sensitive to the distribution of reinforcers across the concurrent schedules. When the procedure required discrimination of the stimulus correlated with each reinforcer, the other 2 subjects also observed the stimuli, and their schedule sensitivity was increased as well. These results implicate procedural differences, rather than inherent behavioral differences, as the source of differences in sensitivity to schedules of reinforcement between humans and nonhumans.     

Control of the behavior of schizophrenic patients by food

Operant-conditioning principles using food as a reinforcer were applied to control the behavior of 45 chronic schizophrenic patients. The investigation was conducted in a psychiatric ward in which there was 24-hr environmental control.

In order to use food as a reinforcer for controlling psychotic behavior, it was necessary first to deal with the eating deficits in the patients. Approximately 50% of the ward population was selected because of a history of refusal to eat. Their refusal to eat had remained relatively unaffected by one or more of these treatments: spoonfeeding, tubefeeding, intravaneous feeding, and electroshock. These treatments were discontinued, and the patients were left alone at mealtimes. The results show that social reinforcement in such forms as coaxing, persuading, and feeding the patient tend to shape patients into eating problems so they are conditioned to eat only with assistance. When refusal to eat was no longer followed by social reinforcement, the patients soon started eating unassisted. When access to the dining room was made dependent upon a chain of responses including a motor and social component, all patients learned these responses.

     

Examining the discriminative and strengthening effects of reinforcers in concurrent schedules

Reinforcers may increase operant responding via a response-strengthening mechanism whereby the probability of the preceding response increases, or via some discriminative process whereby the response more likely to provide subsequent reinforcement becomes, itself, more likely. We tested these two accounts. Six pigeons responded for food reinforcers in a two-alternative switching-key concurrent schedule. Within a session, equal numbers of reinforcers were arranged for responses to each alternative. Those reinforcers strictly alternated between the two alternatives in half the conditions, and were randomly allocated to the alternatives in half the conditions. We also varied, across conditions, the alternative that became available immediately after a reinforcer. Preference after a single reinforcer always favored the immediately available alternative, regardless of the local probability of a reinforcer on that alternative (0 or 1 in the strictly alternating conditions, .5 in the random conditions). Choice then reflected the local reinforcer probabilities, suggesting some discriminative properties of reinforcement. At a more extended level, successive same-alternative reinforcers from an alternative systematically shifted preference towards that alternative, regardless of which alternative was available immediately after a reinforcer. There was no similar shift when successive reinforcers came from alternating sources. These more temporally extended results may suggest a strengthening function of reinforcement, or an enhanced ability to respond appropriately to "win-stay" contingencies over "win-shift" contingencies.