The emergence of conditioned reinforcement from observation

We report an experiment in which observations of peers by six 3-5-year-old participants under specific conditions functioned to convert a small plastic disc or, for one participant, a small piece of string, from a nonreinforcer to a reinforcer. Prior to the observational procedure, we compared each participant's responding on (a) previously acquired performance tasks in which the child received either a preferred food item or the disc (string) for correct responses, and (b) the acquisition of new repertoires in which the disc (string) was the consequence for correct responses. Verbal corrections followed incorrect responses in the latter tasks. The results showed that discs and strings did not reinforce correct responses in the performance tasks, but the food items did; nor did the discs and strings reinforce correct responses in learning new repertoires. We then introduced the peer observation condition in which participants engaged in a different performance task in the presence of a peer who also performed the task. A partition blocked the participants from seeing the peers' performance. However, participants could observe peers receiving discs or strings. Participants did not receive discs or strings regardless of their performance. Peer observation continued until the participants either requested discs or strings repeatedly, or attempted to take discs or strings from the peers. Following the peer observation condition, the same performance and acquisition tasks in which participants had engaged prior to observation were repeated. The results showed that the discs and strings now reinforced correct responding for both performance and acquisition for all participants. We discuss the results with reference to research involving nonhuman subjects that demonstrated the observational conditioning of reinforcers.     

Establishing books as conditioned reinforcers for preschool children as a function of an observational intervention

We tested the effects of an observational intervention (Greer & Singer-Dudek, 2008) on establishing children's books as conditioned reinforcers using a delayed multiple baseline design. Three preschool students with mild language and developmental delays served as the participants. Prior to the intervention, books did not function as reinforcers for any of the participants. The observational intervention consisted of a situation in which the participant observed a confederate being presented with access to books contingent on correct responses and the participant received nothing for correct responses. After several sessions of this treatment, the previously neutral books acquired reinforcing properties for maintenance and acquisition responses for all three participants.     

Value transmission in discrimination learning involving stimulus chains.

Rats learned a series of reversals of a positional discrimination in which responses to one lever led to delayed food and responses to a second lever led to no food. Interpolated within the delays leading to the different outcomes were two-link stimulus chains. The pairing of each stimulus element with the delayed outcome of food or no food varied across reversals. Either stimulus element could have the same correlation with outcome as occurred on the preceding reversal or the opposite correlation as on the preceding reversal. New reversals were acquired more quickly when both stimulus elements had the same status as during the preceding reversal, and were acquired most slowly when both stimulus elements had the opposite status as that of the preceding reversal. The rate of learning was intermediate when only one of the stimulus elements had the same status as that during the preceding reversal. All of the data are compatible with an interpretation in terms of backward chaining of stimulus value.     

A comparison of procedures for unpairing conditioned reflexive motivating operations

This study compared the effectiveness of two procedures to reduce behavior evoked by a reflexive conditioned motivating operation (CMO‐R). Task demands were shown to evoke escape‐maintained problem behavior for 4 students with disabilities. Alternative communication responses were taught as an appropriate method to request escape and this treatment combined with extinction for problem behavior led to decreases in problem behavior for all students. A beeping timer was then arranged to temporally precede the task demand to create a CMO‐R that evoked communication responses. When data showed that the sound of the timer was functioning as a CMO‐R, two methods to reduce behavior evoked by a CMO‐R—extinction unpairing and noncontingent unpairing—were evaluated. Results indicated that noncontingent unpairing was an effective method to reduce the evocative effects of the CMO‐R. Extinction produced unsystematic effects across participants. Results are discussed in terms of abolishing CMOs and the implications of CMOs.     

Devaluation of stimuli contingent on choice: evidence for conditioned reinforcement

Pigeons were presented a concurrent-chains schedule of reinforcement that had terminal links of equal duration. The initial links of the schedule were periodically interrupted by 15-s periods during which an extinction schedule was in effect. The extinction periods were presented on either a response-contingent or a noncontingent basis. Relative response rate for the left alternative decreased when the extinction periods were accompanied by the left terminal-link stimulus. Relative response rate for the right alternative decreased when the extinction periods were accompanied by the right terminal-link stimulus. Relative response rate varied inversely with the frequency of presentation of the extinction periods but was unaffected by presence versus absence of the response contingency in the schedule of extinction-period presentation. Furthermore, relative response rate was unaffected by presentation of extinction periods accompanied by a novel stimulus. When the extinction periods were presented after reinforcement in the left terminal link instead of as interruptions of the initial links, relative response rate for the left alternative was reduced if the postreinforcement extinction period was accompanied by the terminal-link stimulus for the left chain and reduced less if the extinction period was accompanied by the terminal-link stimulus for the right chain. The results demonstrate that the correlation between the terminal-link stimulus and extinction influenced the relative response rate in the initial link.     

The Effect of Negative Incentives in Serial Learning: VII. Theory of Punishment

Eighty three-, four-, five-, and six-year-old boys and girls were presented with experimental conditions where context, transformational complexity, and verbal cue were varied, Two experimental tasks, a perceptually Concrete and an Abstract task, were included and Ss were assigned to a Verbal Cue or a Nonverbal Cue condition. All Ss responded to both the present progressive and the past tenses. Results revealed significant effects of the Age and Tense factors, and improved performance on Comprehension over Production. Younger children made fewer overall errors than older ones. Ss performed better on the present progressive than on the past tense on the perceptually Concrete task, while the reverse was true on the Abstract task. These findings demonstrate that language performance is the result of a complex interrelationship between transformational complexity and the perceptual properties of linguistic structures.     

Effects of delayed conditioned reinforcement in chain schedules.

The contingency between responding and stimulus change on a chain variable-interval 33-s, variable-interval 33-s, variable-interval 33-s schedule was weakened by interposing 3-s delays between either the first and second or the second and third links. No stimulus change signaled the delay interval and responses could occur during it, so the obtained delays were often shorter than the scheduled delay. When the delay occurred after the initial link, initial-link response rates decreased by an average of 77% with no systematic change in response rates in the second or third links. Response rates in the second link decreased an average of 59% when the delay followed that link, again with little effect on response rates in the first or third links. Because the effect of delaying stimulus change was comparable to the effect of delaying primary reinforcement in a simple variable-interval schedule, and the effect of the unsignaled delay was specific to the link in which the delay occurred, the results provide strong evidence for the concept of conditioned reinforcement.     

Choice with delayed and probabilistic reinforcers: effects of variability, time between trials, and conditioned reinforcers.

In a discrete-trials procedure with pigeons, a response on a green key led to a 4-s delay (during which green houselights were lit) and then a reinforcer might or might not be delivered. A response on a red key led to a delay of adjustable duration (during which red houselights were lit) and then a certain reinforcer. The delay was adjusted so as to estimate an indifference point--a duration for which the two alternatives were equally preferred. Once the green key was chosen, a subject had to continue to respond on the green key until a reinforcer was delivered. Each response on the green key, plus the 4-s delay that followed every response, was called one "link" of the green-key schedule. Subjects showed much greater preference for the green key when the number of links before reinforcement was variable (averaging four) than when it was fixed (always exactly four). These findings are consistent with the view that probabilistic reinforcers are analogous to reinforcers delivered after variable delays. When successive links were separated by 4-s or 8-s "interlink intervals" with white houselights, preference for the probabilistic alternative decreased somewhat for 2 subjects but was unaffected for the other 2 subjects. When the interlink intervals had the same green houselights that were present during the 4-s delays, preference for the green key decreased substantially for all subjects. These results provided mixed support for the view that preference for a probabilistic reinforcer is inversely related to the duration of conditioned reinforcers that precede the delivery of food.     

On the predictive validity of behavioral momentum theory for mitigating resurgence of problem behavior

We summarize the results of four recent translational studies from our lab that used the predictions of behavioral momentum theory to inform the development of more durable treatments for destructive behavior. Treatments informed by behavioral momentum theory generally showed better suppression of target responding during an extinction challenge than did a comparison treatment. We reanalyze data from each of the four studies to show that this general finding is apparent both at the aggregate (i.e., proportion of baseline response rates averaged across participants) and within participant (i.e., percentage reduction in proportion of baseline response rates, difference in raw response rates during the extinction challenge). Interestingly, participants who experienced multiple cycles of the extinction challenge generally showed less differentiation in target responding between the treatment informed by behavioral momentum theory and the comparison treatment. Overall results suggest that applications of behavioral momentum theory can substantially improve the durability of common treatments for destructive behavior.     

Key pecking of pigeons under variable-interval schedules of briefly signaled delayed reinforcement: effects of variable-interval value.

Key pecking of 4 pigeons was maintained under a multiple variable-interval 20-s variable-interval 120-s schedule of food reinforcement. When rates of key pecking were stable, a 5-s unsignaled, nonresetting delay to reinforcement separated the first peck after an interval elapsed from reinforcement in both components. Rates of pecking decreased substantially in both components. When rates were stable, the situation was changed such that the peck that began the 5-s delay also changed the color of the keylight for 0.5 s (i.e., the delay was briefly signaled). Rates increased to near-immediate reinforcement levels. In subsequent conditions, delays of 10 and 20 s, still briefly signaled, were tested. Although rates of key pecking during the component with the variable-interval 120-s schedule did not change appreciably across conditions, rates during the variable-interval 20-s component decreased greatly in 1 pigeon at the 10-s delay and decreased in all pigeons at the 20-s delay. In a control condition, the variable-interval 20-s schedule with 20-s delays was changed to a variable-interval 35-s schedule with 5-s delays, thus equating nominal rates of reinforcement. Rates of pecking increased to baseline levels. Rates of pecking, then, depended on the value of the briefly signaled delay relative to the programmed interfood times, rather than on the absolute delay value. These results are discussed in terms of similar findings in the literature on conditioned reinforcement, delayed matching to sample, and classical conditioning.     

Context specificity of conditioned-reinforcement effects on discrimination acquisition

Pigeons were trained on a series of reversals of a simultaneous form discrimination in which the trial outcomes were separated from the choice responses by an 8-s delay interval. Different conditions were defined by the stimuli occurring during the two halves of the delay interval. Discrimination learning was greatly facilitated by having differential stimuli during the delay following correct versus incorrect choices. When the differential stimuli appeared only at the midpoint of the delay, some facilitation occurred relative to when no different stimuli occurred, but there was substantially less facilitation than when the differential stimuli occurred immediately contingent on choice. A reversed-stimulus condition, in which the stimulus at the onset of the delay following a correct choice was the same as that during the last segment of the delay following an incorrect choice, and the stimulus at the onset of the delay following an incorrect choice was the same as that preceding food during the last segment of the delay following a correct choice, also facilitated discrimination learning relative to the nondifferential stimulus conditions.     

Substitutability between conditioned and primary reinforcers in discrimination acquisition.

Rats and pigeons were trained on a series of reversals of a conditional simultaneous discrimination. The percentage of reinforcement for correct trials was varied across reversals. When nonreinforced correct trials produced the same feedback as incorrect trials, the number of errors to reach an acquisition criterion was greater for smaller percentages of reinforcement, but the number of reinforcers required was either approximately constant or smaller for the smaller percentages. When a stimulus paired with food (the conditioned reinforcer) was added on nonreinforced correct trials, both measures were substantially decreased. When the same stimulus was presented, but without a history of food pairing, learning rate was similar to when no stimulus was presented on nonreinforced trials. The results provide direct evidence that conditioned reinforcers may substitute, although imperfectly, for a primary reinforcer, and that pairing with the primary reinforcer is a necessary condition for such substitutability to occur.     

William Nathan Schoenfeld (1915-1996): innovative scientist, inspiring teacher, relentless questioner, complicated man.

The scientific and academic contributions of the late William N. Schoenfeld (1915–1996) are large and, together with his personal qualities as a researcher, thinker, and teacher, supply the themes for this memorial essay on his work and life.     

The effects of cocaine on behavior maintained by timeout from avoidance.

Rats were trained on concurrent schedules under which responses on one lever postponed shock (avoidance) and responses on the other lever produced brief (2-min) periods of signaled timeout from avoidance. For 6 rats, timeout from avoidance was programmed on a variable-interval 45-s schedule that generally resulted in rates that were lower than those on the avoidance lever. For another 6 rats, timeout was arranged on a variable-ratio 15 schedule that produced higher baseline rates. Cocaine (3 to 40 mg/kg) produced large, dose-dependent increases in behavior maintained by timeout in both groups of rats. Avoidance responding was also generally increased by cocaine, but the increases were of lesser magnitude. Increases in response rates were seen across a broad range of doses on behavior maintained by either interval or ratio schedules, an outcome that was unexpected on the basis of most studies of cocaine on food-maintained behavior. These results were similar to those of previous studies of the effects of amphetamine on behavior maintained by timeout from avoidance and suggest that stimulant drugs affect behavior maintained under a shock-postponement schedule differently than they affect behavior maintained by timeout from avoidance.     

Effects of cocaine on briefly signaled versus completely signaled delays to reinforcement.

Key pecking by 4 pigeons was maintained by a multiple schedule consisting of two variable-interval 60-s schedules wherein each food presentation followed a nonresetting 27-s delay that was either briefly signaled at its outset or completely signaled. Brief-signal duration was adjusted so that response rates maintained by the briefly and completely signaled delays of reinforcement were similar. In general, acute administration of small to intermediate doses (0.3 to 3.0 mg/kg) of cocaine produced either small increases in response rates in both components or no change, and larger doses (5.6 to 13.0 mg/kg) decreased response rates. Chronic (i.e., daily) cocaine administration (10.0 mg/kg) resulted in tolerance to the rate-decreasing effects in both components. Cocaine's effects were generally similar whether delays were completely or briefly signaled. Discontinuation of cocaine administration and subsequent removal of the delay signals also had similar effects in both components of the multiple schedule. Taken together, these results are consistent with the view that the two types of delay signals were equally effective in maintaining responding during the variable-interval schedules.     

Choice with delayed and probabilistic reinforcers: effects of prereinforcer and postreinforcer stimuli.

In Experiment 1, pigeons' pecks on a green key led to a 5-s delay with green houselights, and then food was delivered on 20% (or, in other conditions, 50%) of the trials. Pecks on a red key led to an adjusting delay with red houselights, and then food was delivered on every trial. The adjusting delay was used to estimate indifference points: delays at which the two alternatives were chosen about equally often. Varying the presence or absence of green houselights during the delays that preceded possible food deliveries had large effects on choice. In contrast, varying the presence of the green or red houselights in the intertrial intervals had no effects on choice. In Experiment 2, pecks on the green key led to delays of either 5 s or 30 s with green houselights, and then food was delivered on 20% of the trials. Varying the duration of the green houselights on nonreinforced trials had no effect on choice. The results suggest that the green houselights served as a conditioned reinforcer at some times but not at others, depending on whether or not there was a possibility that a primary reinforcer might be delivered. Given this interpretation of what constitutes a conditioned reinforcer, most of the results were consistent with the view that the strength of a conditioned reinforcer is inversely related to its duration.     

Responding of pigeons under variable-interval schedules of signaled-delayed reinforcement: effects of delay-signal duration.

Two experiments with pigeons examined the relation of the duration of a signal for delay ("delay signal") to rates of key pecking. The first employed a multiple schedule comprised of two components with equal variable-interval 60-s schedules of 27-s delayed food reinforcement. In one component, a short (0.5-s) delay signal, presented immediately following the key peck that began the delay, was increased in duration across phases; in the second component the delay signal initially was equal to the length of the programmed delay (27 s) and was decreased across phases. Response rates prior to delays were an increasing function of delay-signal duration. As the delay signal was decreased in duration, response rates were generally higher than those obtained under identical delay-signal durations as the signal was increased in duration. In Experiment 2 a single variable-interval 60-s schedule of 27-s delayed reinforcement was used. Delay-signal durations were again increased gradually across phases. As in Experiment 1, response rates increased as the delay-signal duration was increased. Following the phase during which the signal lasted the entire delay, shorter delay-signal-duration conditions were introduced abruptly, rather than gradually as in Experiment 1, to determine whether the gradual shortening of the delay signal accounted for the differences observed in response rates under identical delay-signal conditions in Experiment 1. Response rates obtained during the second exposures to the conditions with shorter signals were higher than those observed under identical conditions as the signal duration was increased, as in Experiment 1. In both experiments, rates and patterns of responding during delays varied greatly across subjects and were not systematically related to delay-signal durations. The effects of the delay signal may be related to the signal's role as a discriminative stimulus for adventitiously reinforced intradelay behavior, or the delay signal may have served as a conditioned reinforcer by virtue of the temporal relation between it and presentation of food.     

The molarity of molecular theory and the molecularity of molar theory.

Dinsmoor (2001) rejects shock-frequency reduction as a reinforcer for avoidance behavior, and considers this to be an invalidation of so-called molar avoidance theory. This is a narrow view of operant avoidance theory, for which shock-frequency reduction is by no means the only reinforcer.     

Food-paired stimuli as conditioned reinforcers: effects of d-amphetamine.

Seven pigeons were studied in two experiments in which key pecks were reinforced under a second-order schedule wherein satisfaction of variable-interval schedule requirements produced food or a brief stimulus. In the second part of each session, responses produced only the brief stimulus according to a variable-interval schedule (food extinction). For the 4 pigeons in Experiment 1, the response key was red throughout the session. In separate phases, the brief stimulus was either paired with food, not paired with food, or not presented during extinction. d-Amphetamine (0.3 to 10.0 mg/kg) dose-dependently reduced food-maintained responding during the first part of the session and, at intermediate dosages, increased responding during the extinction portion of the session. The magnitude of these increases, however, did not consistently depend on whether the brief stimulus was paired, not paired, or not presented. It was also true that under nondrug conditions, response rates during extinction did not differ reliably depending on pairing operations for the brief stimulus. In Experiment 2, 3 different pigeons responded under a procedure wherein the key was red in the component with food presentations and blue in the extinction component (i.e., multiple schedule). Again, d-amphetamine produced dose-related decreases in responding during the first part of a session and increases in responding in the second part of the session. These increases, however, were related to the pairing operations; larger increases were observed when the brief stimulus was paired with food than when it was not or when it was not presented at all. Under nondrug conditions, the paired brief stimulus controlled higher response rates during extinction than did a nonpaired stimulus or no stimulus. These findings suggest that d-amphetamine can enhance the efficacy of conditioned reinforcers, and that this effect may be more robust if conditioned reinforcers occur in the context of a signaled period of extinction.