Food-paired stimuli as conditioned reinforcers: effects of d-amphetamine.

Seven pigeons were studied in two experiments in which key pecks were reinforced under a second-order schedule wherein satisfaction of variable-interval schedule requirements produced food or a brief stimulus. In the second part of each session, responses produced only the brief stimulus according to a variable-interval schedule (food extinction). For the 4 pigeons in Experiment 1, the response key was red throughout the session. In separate phases, the brief stimulus was either paired with food, not paired with food, or not presented during extinction. d-Amphetamine (0.3 to 10.0 mg/kg) dose-dependently reduced food-maintained responding during the first part of the session and, at intermediate dosages, increased responding during the extinction portion of the session. The magnitude of these increases, however, did not consistently depend on whether the brief stimulus was paired, not paired, or not presented. It was also true that under nondrug conditions, response rates during extinction did not differ reliably depending on pairing operations for the brief stimulus. In Experiment 2, 3 different pigeons responded under a procedure wherein the key was red in the component with food presentations and blue in the extinction component (i.e., multiple schedule). Again, d-amphetamine produced dose-related decreases in responding during the first part of a session and increases in responding in the second part of the session. These increases, however, were related to the pairing operations; larger increases were observed when the brief stimulus was paired with food than when it was not or when it was not presented at all. Under nondrug conditions, the paired brief stimulus controlled higher response rates during extinction than did a nonpaired stimulus or no stimulus. These findings suggest that d-amphetamine can enhance the efficacy of conditioned reinforcers, and that this effect may be more robust if conditioned reinforcers occur in the context of a signaled period of extinction.     

Summation: Further assessment of a configural theory

Rats received Pavlovian conditioning in which food was signalled by a visual stimulus, A+, an auditory stimulus, B+, and a compound composed of different visual and auditory stimuli, CD+. Test trials were then given with the compound AB. Experiments 1 and 2A revealed stronger responding during AB than during CD. In Experiment 2B, there was no evidence of a summation of responding during AB when A+ B+ training was conducted in the absence of CD+ trials. A further failure to observe abnormally strong responding during ABwas found in Experiment 3 for which the training trials with A+ B+ CD+ were accompanied by trials in which C and D were separately paired with food. The results are explained in terms of a configural theory of conditioning, which assumes that responding during a compound is determined by generalization from its components, as well as from other compounds to which it is similar.     

Inhibitory sensory preconditioning.

In two experiments rats were preexposed to neutral stimuli. Both experiments used a between-subjects design in which a paired group was preexposed to intermixed presentations of A --> Band AX, and an unpaired control group was preexposed to intermixed presentations of A, B, and AX. After the conditioning of B, in Experiment 1, conditioned responding to X was acquired more slowly in the paired than in the unpaired group. Furthermore, in Experiment 2, X reduced conditioned responding to a separately trained excitor in a summation test but only in the paired group. Together, these results provide evidence of an inhibitory form of sensory preconditioning.     

Disruption of responding maintained by conditioned reinforcement: alterations in response-conditioned-reinforcer relations

An observing procedure was used to investigate the effects of alterations in response-conditioned-reinforcer relations on observing. Pigeons responded to produce schedule-correlated stimuli paired with the availability of food or extinction. The contingency between observing responses and conditioned reinforcement was altered in three experiments. In Experiment 1, after a contingency was established in baseline between the observing response and conditioned reinforcement, it was removed and the schedule-correlated stimuli were presented independently of responding according to a variable-time schedule. The variable-time schedule was constructed such that the rate of stimulus presentations was yoked from baseline. The removal of the observing contingency reliably reduced rates of observing. In Experiment 2, resetting delays to conditioned reinforcement were imposed between observing responses and the schedule-correlated stimuli they produced. Delay values of 0, 0.5, 1, 5, and 10 s were examined. Rates of observing varied inversely as a function of delay value. In Experiment 3, signaled and unsignaled resetting delays between observing responses and schedule-correlated stimuli were compared. Baseline rates of observing were decreased less by signaled delays than by unsignaled delays. Disruptions in response-conditioned-reinforcer relations produce similar behavioral effects to those found with primary reinforcement.     

Punishment of schedule-induced drinking in rats by signaled and unsignaled delays in food presentation

Food-deprived rats were exposed to a fixed-time 60-s schedule of food-pellet presentation and developed schedule-induced drinking. Using an ABA reversal design, three experiments investigated the effects of events then made dependent on licks. In Experiment 1, lick-dependent signaled delays (10 s) in food presentation in general led to decreased drinking, which recovered when the signaled delays were discontinued. The drinking of yoked-control rats, which received food at the same times as those exposed to the signaled-delay contingency, showed much smaller changes. Experiment 2 showed that 10-s lick-dependent signals alone did not reduce drinking. In Experiment 3, when licks produced unsignaled 10-s delays in food there were less marked and more gradual changes in drinking than in Experiment 1, although these effects again were greater than with yoked-control animals. We concluded that both signaled and unsignaled delays functioned as punishers of drinking. These findings support the view that schedule-induced drinking, like operant behavior, is subject to control by its consequences.     

Overshadowing and associability change

Three appetitive Pavlovian conditioning experiments with rats examined the associability of stimuli A and B that had a history of compound conditioning (AB+), relative to stimuli X and Y that had a history of conditioning in isolation (X+, Y+). Following this training, Experiment 1 revealed that conditioned responding was higher to X and Y than to A and B (overshadowing). In a subsequent AY+, AX-, BY- test discrimination, the AY/BY discrimination was solved more readily than the AY/AX discrimination. In Experiment 2, following AB+, X+, Y+ training, A and Y were presented as a compound and signaled the availability of reinforcement upon the performance of an instrumental response. Test trials in which A and Y were presented alone, and in extinction, revealed that A acquired greater control of instrumental responding than Y. Experiment 3 revealed that following AB+, X+, Y+ training, A and B served as more effective discriminative stimuli for instrumental responding than X and Y. Overall, these results imply that the associability of stimuli conditioned in compound is higher than stimuli conditioned in isolation. These results are discussed in terms of attentional theories of associative learning.     

Timeout postponement without increased reinforcement frequency

Three experiments were conducted to examine pigeons' postponement of signaled extinction periods (timeouts) from a schedule of food reinforcement when such responding neither decreased overall timeout frequency nor increased the overall frequency of food reinforcement. A discrete-trial procedure was used in which a response during the first 5 s of a trial postponed an otherwise immediate 60-s timeout to a later part of that same trial but had no effect on whether the timeout occurred. During time-in periods, responses on a second key produced food according to a random-interval 20-s schedule. In Experiment 1, the response-timeout interval was 45 s under postponement conditions and 0 s under extinction conditions (responses were ineffective in postponing timeouts). The percentage of trials with a response was consistently high when the timeout-postponement contingency was in effect and decreased to low levels when it was discontinued under extinction conditions. In Experiment 2, the response-timeout interval was also 45 s but postponement responses increased the duration of the timeout, which varied from 60 s to 105 s across conditions. Postponement responding was maintained, generally at high levels, at all timeout durations, despite sometimes large decreases in the overall frequency of food reinforcement. In Experiment 3, timeout duration was held constant at 60 s while the response-timeout interval was varied systematically across conditions from 0 s to 45 s. Postponement responding was maintained under all conditions in which the response-timeout interval exceeded 0 s (the timeout interval in the absence of a response). In some conditions of Experiment 3, which were designed to control for the immediacy of food reinforcement and food-correlated (time-in) stimuli, responding postponed timeout but the timeout was delayed whether a response occurred or not. Responding was maintained for 2 of 3 subjects, suggesting that behavior was negatively reinforced by timeout postponement rather than positively reinforced by the more immediate presentation of food or food-correlated (time-in) stimuli.     

Intertrial interval as a contextual stimulus: further analysis of a novel asymmetry in temporal discrimination learning

Four experiments investigated discrimination learning when the duration of the intertrial interval (ITI) signaled whether or not the next conditional stimulus (CS) would be paired with food pellets. Rats received presentations of a 10-s CS separated half the time by long ITIs and half the time by short ITIs. When the long ITI signaled that the CS would be reinforced and the short interval signaled that it would not be (Long+/Short-), rats learned the discrimination readily. However, when the short ITI signaled that the CS would be reinforced and the long interval signaled that it would not (Short+/Long-), discrimination learning was much slower. Experiment 1 compared Long+/Short- and Short+/Long- discrimination learning with 16-min/4-min or 4-min/1-min ITI combinations. Experiment 2 found no evidence that Short+/Long- learning is inferior because the temporal cue corresponding to the short interval is ambiguous. Experiment 3 found no evidence that Short+/Long- learning is poor because the end of a long ITI signals a substantial reduction in delay to the next reinforcer. Long+/Short- learning may be faster than Short+/Long-because elapsing time involves exposure to a sequence of hypothetical stimulus elements (e.g., A then B), and feature-positive discriminations (AB+/A-) are learned quicker than feature-negative discriminations (A+/AB-). Consistent with this view, Experiment 4 found a robust feature-positive effect when sequentially presented CSs played the role of elements A and B.     

Acquired equivalence and distinctiveness of cues

In Experiments 1 and 2, rats received initial training in which two stimuli (A and N) were either followed by the same consequence (food) or by different consequences (food and no food). Subsequently N was paired with electric shock and the generalization of conditioned suppression to A was assessed. Suppression to A was more marked when A and N had both been followed by food than when they had had different outcomes. In Experiment 3, 3 stimuli (A, B, and N) were presented initially. For one group, A and N were paired with food and B was nonreinforced: for a second group, B was paired with food and A and N were nonreinforced. Generalization of suppression was found to be more substantial to A than to B for both groups. These results indicate that the extent to which stimuli are treated as being equivalent is partly determined by their reinforcement histories.     

Responding of pigeons under variable-interval schedules of unsignaled, briefly signaled, and completely signaled delays to reinforcement

In Experiment 1, three pigeons' key pecking was maintained under a variable-interval 60-s schedule of food reinforcement. A 1-s unsignaled nonresetting delay to reinforcement was then added. Rates decreased and stabilized at values below those observed under immediate-reinforcement conditions. A brief stimulus change (key lit red for 0.5 s) was then arranged to follow immediately the peck that began the delay. Response rates quickly returned to baseline levels. Subsequently, rates near baseline levels were maintained with briefly signaled delays of 3 and 9 s. When a 27-s briefly signaled delay was instituted, response rates decreased to low levels. In Experiment 2, four pigeons' responding was first maintained under a multiple variable-interval 60-s (green key) variable-interval 60-s (red key) schedule. Response rates in both components fell to low levels when a 3-s unsignaled delay was added. In the first component delays were then briefly signaled in the same manner as Experiment 1, and in the second component they were signaled with a change in key color that remained until food was delivered. Response rates increased to near baseline levels in both components, and remained near baseline when the delays in both components were lengthened to 9 s. When delays were lengthened to 27 s, response rates fell to low levels in the briefly signaled delay component for three of four pigeons while remaining at or near baseline in the completely signaled delay component. In Experiment 3, low response rates under a 9-s unsignaled delay to reinforcement (tandem variable-interval 60 s fixed-time 9 s) increased when the delay was briefly signaled. The role of the brief stimulus as conditioned reinforcement may be a function of its temporal relation to food, and thus may be related to the eliciting function of the stimulus.     

Effects of signaling on temporal control of behavior in response‐initiated fixed intervals

Behavior and events distributed in time can serve as markers that signal delays to future events. The majority of timing research has focused on how behavior changes as the time to some event, usually food availability, decreases. The primary objective of the two experiments presented here was to assess how behavior changes as time passes between two time markers when the first time marker was manipulated but the second, food delivery, was held constant. Pigeons were exposed to fixed‐interval, response‐initiated fixed‐interval, and signaled response‐initiated fixed‐interval 15‐ and 30‐s schedules of reinforcement. In Experiment 1, first‐response latencies were systematically shorter in the signaled response‐initiated schedules than response‐initiated schedules, suggesting that the first response was a more effective time marker when it was signaled. In Experiment 2, responding in no‐food (i.e. “peak”) trials indicated that timing accuracy was equivalent in the three schedule types. Compared to fixed interval schedules, timing precision was reduced in the signaled response‐initiated schedules and was lowest in response‐initiated schedules. Results from Experiments 1 and 2 coupled with previous research suggest that the overall “informativeness” of a time marker relative to other events and behaviors in the environment may determine its efficacy.     

Blocking between occasion setters and contextual stimuli

Two appetitive conditioning experiments with rats used a blocking procedure to compare the mechanisms through which contexts and positive occasion setters control responding to conditioned stimuli (CSs) in discrimination learning. In Experiment 1, a light (L) initially set the occasion for food reinforcement of a tone CS (T). Then a compound of L and a novel context signaled reinforcement of T. Previous learning about L blocked contextual control of responding to T. Blocking was not due to simple excitation conditioned to L during discrimination training; comparable excitation to L in a control group did not result in blocking. Conversely, in Experiment 2, initial learning about the context blocked the acquisition of occasion setting to L. Excitation conditioned to the context could not account for the blocking. In general, the results suggest that contexts and occasion setters may control responding to CSs through similar mechanisms.     

Signaled Delay of Reinforcement: Effects of Postconditioning Manipulation of Context Associative Strength on Instrumental Performance

Two experiments examined the influence of postconditioning treatments of contextual cues on instrumental responding acquired with a signaled delay of reinforcement schedule. In Experiment 1, mere exposure to the conditioning context after instrumental training resulted in an attenuated response rate during an extinction test. In the second experiment, responding was decreased by exposure to the contextual cues or sessions in which signaled noncontingent reinforcements occurred. The greatest response decrement, however, occurred following unsignaled noncontingent food presentations. The results are discussed with respect to the different roles of contextual cues on operant responding.     

Quasireinforcement: Control of behavior by second-order interval schedules

Pigeons were trained on a variable-interval 66-sec schedule of reinforcement that was segmented into either fixed- or variable-interval 10-sec components. Three-second access to food followed some components according to the overall VI 66-sec schedule, but 3-sec periods of nonreinforcement followed the other components. With both FI 10-sec and VI 10-sec segments, overall response rates were generally higher when the completion of unreinforced segments was signaled by a red key (never paired with food) than when it was unsignaled. Response rates during the red-key periods dropped to zero. Brief presentations of the red key engendered the distinctive (FI or VI) patterns of responding which would be expected if each segment were followed by food. These data demonstrate behavioral control by brief stimuli which are not paired with primary reinforcement and show that such control may develop even when the sequences of behavior required to produce food or brief stimuli are variable in duration.     

Sensitivity and bias under conditions of equal and unequal academic task difficulty

We conducted an experimental analysis of children's relative problem-completion rates across two workstations under conditions of equal (Experiment 1) and unequal (Experiment 2) problem difficulty. Results were described using the generalized matching equation and were evaluated for degree of schedule versus stimulus control. Experiment 1 involved a symmetrical choice arrangement in which the children could earn points exchangeable for rewards contingent on correct math problem completion. Points were delivered according to signaled variable-interval schedules at each workstation. For 2 children, relative rates of problem completion appeared to have been controlled by the schedule requirements in effect and matched relative rates of reinforcement, with sensitivity values near 1 and bias values near 0. Experiment 2 involved increasing the difficulty of math problems at one of the workstations. Sensitivity values for all 3 participants were near 1, but a substantial increase in bias toward the easier math problems was observed. This bias was possibly associated with responding at the more difficult workstation coming under stimulus control rather than schedule control.     

Examining stimuli paired with alternative reinforcement to mitigate resurgence in children diagnosed with autism spectrum disorder and pigeons

In two laboratory experiments, we examined whether stimuli paired with alternative reinforcers could mitigate resurgence of a previously reinforced target response with pigeons (Experiment 1) and children diagnosed with Autism Spectrum Disorder (Experiment 2). In Phase 1, we arranged food reinforcement according to a variable-ratio schedule for engaging in a target response. In Phase 2, we arranged extinction for target responding and differentially reinforced alternative responding according to a fixed-ratio schedule, with every alternative-reinforcer delivery paired with a change in keylight color (Experiment 1) or automated verbal (praise) statement (Experiment 2). In Phase 3, we assessed resurgence during extinction of target and alternative responding in the presence versus absence of continued presentation of the paired stimulus. Despite variation across sessions, resurgence on average was lower when continuing to present the paired stimuli in all pigeons and children while maintenance of alternative responding did not differ between assessments. These findings indicate that stimuli paired with alternative reinforcement can modestly decrease resurgence, but further examination of their efficacy and a better understanding of the underlying processes are necessary before they can be recommended for clinical use in reducing resurgence of clinically relevant problem behavior.     

Occasion-setting training renders stimuli more similar: Acquired equivalence between the targets of feature-positive discriminations

In two experiments rats received training on two concurrent appetitive feature-positive discriminations. A preliminary test in Experiment 1 confirmed previous demonstrations of the transfer of occasion-setting properties—the feature from one of these discriminations was better able to facilitate responding to the occasion-set target CS from the second discrimination than to a control stimulus that had not been the subject of occasion-setting. The source of this transfer was investigated in a second phase of training, and in Experiment 2. In both experiments one of the occasion-set CSs was paired with food, and generalization of appetitive conditioned responding from this stimulus to the second occasion-set CS, and to a control cue, was examined. There was more generalization from the first occasion-set CS to the second CS that had also been occasion-set than to the control cue. This is taken as evidence that occasion-set CSs are rendered more similar as a result of their common training history. The implications of these findings for explaining transfer of occasion setting are discussed.     

Blocking, unblocking, and overexpectation in autoshaping with pigeons

Three experiments used pigeons in an autoshaping procedure and a single-subject design to examine compound stimulus control in classical conditioning. Experiment 1 examined the blocking effect, and Experiment 2 examined the unblocking effect. In both experiments, response-independent food was first delivered intermittently in the presence of one distinctively colored houselight but not another. Then, conventional autoshaping trials were carried out in the presence of each houselight. In Experiment 1, the keylight readily elicited responding in the presence of the houselight that had been negatively correlated with food, but not in the presence of the houselight that had been positively correlated with food. In Experiment 2, the keylight readily elicited responding in the presence of the houselight positively correlated with food, but only when the amount of food used on the autoshaping trials was either greater or less than that previously delivered in the presence of the houselight. Experiment 3 examined the overexpectation effect. Conventional autoshaping trials were first carried out by presenting each of two keylights individually. Then, additional autoshaping trials were carried out by presenting the two keylights as a compound, with either the same amount of food or a greater amount of food per trial. Finally, the keylights were retested by again presenting them individually. The number of responses per trial elicited by the keylights decreased when the amount of food used in compound trials was the same as that used in individual trials. However, the number of responses per trial remained approximately the same when the amount of food used in compound trials was greater than that used in individual trials. Taken together, the results of the three experiments demonstrate (a) the generality of the blocking, unblocking, and overexpectation effects by virtue of their extension to appetitive unconditioned stimuli; (b) the suitability of pigeons as subjects and autoshaping as a procedure for studying classical conditioning; and (c) the appropriateness of single-subject designs.     

Blocking of occasion setting in feature-positive discriminations

In two experiments rats received feature-positive discrimination training in which brief conditioned stimuli (CSs) were paired with food during presentations of an extended feature stimulus, and non-reinforced in its absence. In Experiment 1 a novel feature was trained in compound with a second, pretrained feature. Acquisition of control over responding to the CS by the novel feature was blocked if the pretrained feature had also been trained in a feature-positive discrimination, compared to a group for whom the pretrained feature had been accompanied by uncorrelated presentations of CSs and food. Experiment 2 employed a within-subjects design. It demonstrated that the feature from a feature-positive discrimination with a particular CS, x, blocked acquisition of control by an added, novel feature over responding to x, compared to the control acquired by the same novel feature over a novel, CS y.     

Effect of signaled reinforcement on response variability

Three experiments examined the effect of signaling reinforcement on rats' lever pressing on contingencies that reinforced variable responding to extend the exploration of signaled reinforcement to a schedule that has previously not been examined in this respect. In Experiment 1, rats responding on a lag-8 variability schedule with signaled reinforcement displayed greater levels of variability (U values) than rats on the same schedule lacking a reinforcement signal. In Experiment 2, rats responding on a differential reinforcement of least frequent responses schedule also displayed greater operant variability with a signal for reinforcement compared with rats without a reinforcement signal. In Experiment 3, a reinforcement signal decreased the variability of a response sequence when there was no variability requirement. These results offer empirical corroboration that operant variability responds to manipulations in the same manner as do other forms of operant response and that a reinforcement signal facilitates the emission of the required operant.