An evaluation of the associative strength of a partially reinforced serial CS

Three experiments examined the factors that are responsible for producing a high rate of autoshaped keypecking in pigeons when the conditioned stimulus is followed by varying consequences. In the first experiment, a single group received three serial conditioning schedules presented on different keys. The AB⁺ schedule consisted of trials in which A was followed by B, which, in turn, was followed by food. The CD⁺C° schedule consisted of reinforced CD pairings plus presentations of C alone. The EF^+/0 schedule consisted of a mixture of reinforced and nonreinforced EF pairings. It was found that responding was most rapid during C, slowest during E, and at an intermediate rate during A. In spite of this ordering A was chosen in preference to C, when these stimuli were presented simultaneously. In Experiments 2 and 3 the schedules AB⁺A° and AB^+/0 were presented to separate groups. The rate of responding during A was substantially greater in the former schedule, yet it was found that this stimulus was equally ineffective as a conditioned reinforcer in both groups. We propose that autoshaped keypecking is composed of two types of response, a conditioned response and an orienting response, which are governed by different processes.     

Partial reinforcement in serial autoshaping: The role of attentional and associative factors

In Experiment 1, pigeons were autoshaped to two stimuli (A and B) each followed by delayed reinforcement on 50% of trials. A different stimulus (X) was presented in the interval between stimulus A and reinforcement (A-X-US) and not when stimulus A was nonreinforced (A---). The X stimulus was also presented following the stimulus B trials that were nonreinforced (B-X) and was absent when stimulus B was reinforced (B---US). Not only was the response rate to A higher, but so was the rate in the intervals immediately following A, whether or not these intervals contained X. This latter finding is inconsistent with the interpretation that pigeons discriminated between X when it followed A and when it followed B and leaves open the possibility that X acts as a “catalyst”. The same general design was employed in Experiment 2 as was used in Experiment 1 with the exception that the catalytic function of X was equated for A and B by presenting reinforcement on B-X trials. The A stimulus, however, still generated the higher response rate. A new interpretation is offered that suggests that variable reward magnitude will maintain attention to a stimulus and that attention will generate keypecking.     

Thermoregulation and performance of heat-reinforced autoshaped keypecking in chicks

Performance of autoshaped keypecking reinforced by heat was found to increase and then to decrease when chicks were exposed to training under a constant thermal load on a daily basis over the first 1-2 weeks after hatching. The decline in performance was not affected by variation in US duration (Experiment 1). No decline in performance was observed when food reinforcement was used (Experiment 2), suggesting that the effect depended on the use of thermal motivation. When thermoregulation was disrupted by pharmacological blockade using propranolol hydrochloride (10 mg/kg), keypecking was reinstated (Experiment 3). The increase in keypecking produced by propranolol was not due to a nonspecific energization of behavior inasmuch as the drug depressed autoshaped keypecking established with food reinforcement (Experiment 4). These findings support the conclusion that the decline in autoshaped keypecking reinforced by heat is due to development of the chick's thermoregulatory system. Various physical changes (e.g., feathering) and the increasing effectiveness of autonomic and behavioral thermoregulatory responses reduce the net motivational value of external heat sources, thus leading to a decline in the likelihood of approach and contact with signals for heat.     

A partial reinforcement extinction effect despite equal rates of reinforcement during Pavlovian conditioning

In 4 experiments rats received appetitive Pavlovian conditioning followed by extinction. Food accompanied every trial with the conditioned stimulus (CS) for the continuously reinforced groups and only half of the trials for the partially reinforced groups. In contrast to previous experiments that have compared the effects of partial and continuous reinforcement, the rate at which food was delivered during the CS was the same for both groups. The strength of the conditioned response during extinction weakened more rapidly in the continuously than in the partially reinforced groups. The results demonstrate that the partial reinforcement extinction effect is a consequence of the nonreinforced trials with the CS, rather than the rate at which the unconditioned stimulus is delivered during the CS.     

Within-subject Partial Reinforcement Extinction Effect in Autoshaping

Three autoshaping experiments explored the extinction of various keylight stimuli conditioned with partial and continuous reinforcement within the same pigeon subjects. Extinction of autoshaped keypecking proceeded more slowly in a stimulus trained with 50% (Experiment 1) and 25% (Experiment 2) reinforcement compared with a stimulus given 100% reinforcement. Experiment 3 found conditioning with 75% reinforcement to generate more rapid extinction than that with 25% reinforcement. These results have both theoretical and methodological implications for the study of extinction.     

Choice as a dependent measure in autoshaping: sensitivity to frequency and duration of food presentation.

Previous investigations have shown that rate, latency, and percentage of trials with at least one response are somewhat insensitive measures of the strength of autoshaped responding. In the present studies, these measures were contrasted with the allocation of responding during simultaneous choice tests, a measure of response strength frequently used in operant paradigms. In two experiments, nine pigeons were exposed to a forward pairing autoshaping procedure. Training sessions consisted of the successive presentation of three stimuli, each followed by food on either 100%, 50%, or 0% of the trials. Choice testing involved the simultaneous presentation of the three stimuli. In Experiment I, all pigeons consistently directed their initial choice responses and the majority of subsequent responses to the stimulus always followed by food, despite the fact that during training sessions the response rates of most birds were highest in the presence of the stimulus followed by food on 50% of the trials. In Experiment II, rate, latency, and percentage of trials with at least one response did not change appreciably as a function of duration of feeder presentations. However, choice responding was lawfully affected by duration of feeder presentations. These data suggest that choice is perhaps a more sensitive measure of the strength of autoshaped responding than other, more commonly employed, indices.     

Searching for Evidence of Transfer between Drug Facilitators

Four autoshaping experiments with pigeons examined the transfer of facilitation between drug stimuli. In Experiment 1 the presence and absence of methadone or phencyclidine signaled which of two keylight CSs were followed by grain (US). A different keylight CS was reinforced in the absence of the drugs. The drugs facilitated responding to the CS with which they were trained but did not facilitate responding to the CS reinforced in the presence of the other drug. Thus, transfer of facilitation across drug stimuli was not obtained. In Experiment 2 one of the drug facilitators was replaced by a tone. No evidence of transfer of facilitative function was obtained between the tone and the remaining drug facilitator. In Experiment 3 transfer was again not found using a design similar to that used in Experiment 1 except that the drugs did not simultaneously signal nonreinforcement of a keylight CS. In Experiment 4 the procedures of Experiment 1 were employed except that drug facilitators were replaced by distinct visual stimuli surrounding the response key. Clear evidence of transfer between the visual facilitators was obtained. The lack of transfer of modulatory control by drug stimuli in Experiments 1-3, coupled with its occurrence in Experiment 4 and previous studies, suggests that stimulus control by drugs may differ in important ways from stimulus control by conventional stimuli.     

Determinants of Response Recovery in Extinction Following Response Elimination

Elimination of autoshaped responding by random or negatively-contingent response-reductive procedures may leave a previously acquired association intact; however, two previous studies suggest that response elimination by backward conditioning may have more permanent effects. In Experiments 1 and 3, the autoshaped responding of pigeons was eliminated by a backward or negative contingency. Although speed of recovery during a subsequent extinction phase was greater following the backward contingency in Experiment 3, level of recovery did not differ as a function of response elimination procedure in either experiment. A possible basis for the discrepancy between the present and previous findings is the contingency arranged between the conditioned and unconditioned stimuli (CS and US, respectively) during forward conditioning. In Experiments 1 and 2, probability of US presentation following the CS (p(US|CS) = 0.4 or 1.0) was varied within (Experiment 1) or between (Experiment 2) groups of birds. In both cases, level of recovery was higher following training under (p(US|CS) = 0.4). The results are consistent with a memory retrieval theory in which unreinforced trials function as a retrieval cue for not responding.     

Autoshaping with common and distinctive stimulus elements, compact and dispersed arrays

Four groups of pigeons were trained with a standard autoshaping procedure in which a brief fixed-duration interval always followed by a grain delivery alternated with a longer variable-duration interval never associated with grain delivery. One of two stimuli was always presented during each interval. One of them contained three black dots and a black star on a green background; the other contained four black dots on a green background. The four elements of each stimulus were arranged in a more compact array for two groups and in a more dispersed array for the other two groups. Which of the two stimuli preceded grain delivery was counterbalanced within each pair of groups. The speed of occurrence of the first autoshaped peck was not affected by whether the stimulus containing the distinctive star element preceded grain delivery, but autoshaping was faster when the stimulus arrays were compact than when they were dispersed. During 560 response-independent training trials that followed the first autoshaped peck, this pattern reversed; both discriminative control over responding and the relative frequency of pecking the stimulus that preceded grain delivery were greater for the two groups where this stimulus contained the discriminative element than for the two groups where it contained only common elements. During subsequent testing with stimuli containing only a single element each, the distinctive feature was responded to proportionately more often by the two groups for which it had been an element of the stimulus preceding grain delivery than by the two groups for which it had been an element of the stimulus complex that never was associated with grain delivery. These data add further support to the hypothesis that the initial occurrence of autoshaped responding and its subsequent maintenance are not affected by the same variables. They also suggest that automaintenance is as sensitive as response-dependent training to the presence or absence of a distinctive stimulus element among several common elements and that this sensitivity appears to be independent of the specific method used for presenting the stimuli during automaintenance.     

Blocking with serial compound stimuli: The role of local context and second-order associations

Blocking was studied with rats in two serial conditioning experiments in which CS1 was followed by CS2 and then shock. Experiment 1 demonstrated that pretraining with CS1 was able to block conditioning to CS2 when the pretraining consisted of trace conditioning. But when serial conditioning was used for pretraining, with a third stimulus as the second element of the compound, then blocking was not detected during the subsequent phase. In Experiment 2 the effect of pretraining with CS2 on blocking with CS1 was examined. Blocking was effective, but only when steps were taken to minimize the growth of second-order associations resulting from the pairing of CS1 with CS2. These results are consistent with a principle stating that the ability of a pretrained stimulus to block the added stimulus in a compound depends on the relative contiguity of each stimulus to the US.     

Forward and backward second-order Pavlovian conditioning in honeybees

Second-order conditioning (SOC) is the association of a neutral stimulus with another stimulus that had previously been combined with an unconditioned stimulus (US). We used classical conditioning of the proboscis extension response (PER) in honeybees (Apis mellifera) with odors (CS) and sugar (US). Previous SOC experiments in bees were inconclusive, and, therefore, we attempted to demonstrate SOC in the following three experiments: (Experiment 1) After differential conditioning (pairing odor A with US and presenting odor B without US), the bees experienced two pairs of partially overlapping odors, either a new odor C followed by a previously reinforced odor A (C-A) or a new odor C followed by a previously nonreinforced odor B (C-B). (Experiment 2) After differential conditioning, bees were presented with C-A or A-C. (Experiment 3) Bees were first presented with C-A or A-C before differential conditioning and were tested with odor C. We observed: (Experiment 1) 40% of the bees showed PER to the C-A presentation, but only 20% showed PER to the C-B presentation. (Experiment 2) 40% of the bees showed PER to the C-A presentation, while only 20% showed PER to the reversed sequence A-C. Experiments 1 and 2 showed that a previously reinforced odor can be a secondary reinforcer for excitatory SOC only with forward-pairing. (Experiment 3) PER toward C was lower (15%) in bees presented with A-C than with C-A (25%). This showed that backward SOC is not as effective as forward SOC. These results help to delineate different conditions that are critical for the phenomenon of SOC.     

Pavlovian contingencies and temporal information

The effects of altering the contingency between the conditioned stimulus (CS) and the unconditioned stimulus (US) on the acquisition of autoshaped responding was investigated by changing the frequency of unsignaled USs during the intertrial interval. The addition of the unsignaled USs had an effect on acquisition speed comparable with that of massing trials. The effects of these manipulations can be understood in terms of their effect on the amount of information (number of bits) that the average CS conveys to the subject about the timing of the next US. The number of reinforced CSs prior to acquisition is inversely related to the information content of the CS.     

Differential Conditioning Based on a Difference in the Predictability of Reinforcement

A Pavlovian procedure containing a set of several alternative reliable trial stimuli and a set of several alternative unreliable trial stimuli was used to determine if reliability of outcome, per se, would differentially condition additional trial stimuli that either preceded or followed those original trial stimuli in a serial conditioning paradigm. The reinforcing effectiveness of the antecedent stimuli themselves was later tested in a higher-order conditioning design. It was found that different rates could be established to the different antecedent stimuli in the absence of a difference in the probability of reinforcement. Further, stimuli paired with those antecedent stimuli in the higher-order conditioning test also exhibited a difference in rate. It was concluded that a combination of a stimulus that is always followed by food and another that is never followed by food are, in net, a more effective reinforcer than a common stimulus that has the same overall probability of reinforcement but which is followed by both outcomes.     

Contextual control of appetitive conditioning: Influence of a contextual stimulus generated by a partial reinforcement procedure

Three experiments with rat subjects examined the effects of a context switch after conditioning treatments in which the conditioned stimulus (CS) was either paired with food on every presentation (continuous reinforcement) or on some of its presentations (partial reinforcement). In each experiment, a target CS was given one of these treatments in Context A, and another CS was given a treatment during sessions that were intermixed in Context B. Final tests of the target CS in Context A and Context B often revealed no loss of responding with the switch to B. However, a loss was observed when partial reinforcement had been associated with Context A and continuous reinforcement had been associated with Context B. Those conditions caused equal decrements in responding to partially reinforced and continuously reinforced targets. The results suggest that under the present conditions partial reinforcement can generate a contextual stimulus that becomes associated with the physical context and controls responding to the CS.     

Autoshaping in the rat: Effects of omission on the form of the response

Two experiments were conducted to investigate the effect of an omission contingency on behavior related to and characterizing autoshaped lever contacts in the rat. In Experiment I an omission contingency imposed on autoshaped lever contacts forceful enough to produce a press (.078N) resulted in a significant decrease in lever presses, but had no effect on frequency of lever touches (contacts of insufficient force to produce a press) or rate of food tray entry during lever presentation. In contrast, rats which received a similar number of lever-food pairings, but whose behavior had no programmed consequences (yoked control subjects), showed an increase in lever press rate, a significant decrease in rate of food tray entry, and no change in rate of lever-touches. In Experiment II, the effect of a similar omission contingency on the topography of lever contact responses was investigated. Prior to omission training subjects contacted the lever primarily by pawing it. Following omission training this behavior was suppressed, with a subsequent increase in lever contacts characterized as nosing. Yoked control subjects showed no significant changes in lever contact topography. The results indicate that (1) an omission contingency does not simply eliminate wholesale those topographies which incur the contingency but produces subtle adaptive changes in lever contact topography; and (2) the nature of the autoshaped response in the rat does not appear to be rigid enough to depend solely upon the nature of the unconditioned stimulus or the conditioned stimulus, but can also be determined by the relationships existing between the animal's behavior and these stimuli.     

Occasional reinforced trials during extinction can slow the rate of rapid reacquisition

Two appetitive conditioning experiments with rats examined reacquisition after conditioned responding was eliminated by either extinction or by a partial reinforcement procedure in which reinforced trials were occasionally presented among many nonreinforced trials. In Experiment 1, reacquisition to a conditional stimulus (CS) that had been conditioned and extinguished was more rapid than acquisition in a group that had received no prior conditioning. However, the addition of occasional reinforced trials to extinction slowed this rapid reacquisition effect. Experiment 2 replicated the result and showed that a procedure in which the CS and the unconditional stimulus (US) were unpaired in extinction interfered even further with reacquisition. The results suggest that rapid reacquisition is ordinarily produced when reinforced trials provide a contextual cue that can renew responding by signaling other acquisition trials (Ricker & Bouton, 1996). The effects of partial reinforcement in extinction are surprising from several theoretical perspectives and have useful clinical implications.     

Blocking acquisition of the rabbit's nictitating membrane response to serial conditioned stimuli

Three experiments demonstrated that prior training with one stimulus (CS1) would block acquisition of the rabbit's nictitating membrane response (NMR) to another stimulus (CS2) which was more contiguous to the US during serial compound training (CS1-CS2-US). Specifically, the CS1-US interval was 800 msec, which produces only a modest rate of CR acquisition, while the CS2-US interval was 400 msec, which is an optimal value for the NMR preparation. Experiment 1 demonstrated blocking when CS1 overlapped CS2, and Experiment 3 demonstrated blocking when CS1 and CS2 were presented in a strictly sequential fashion. Experiment 2 showed that the magnitude of blocking in the serial compound was comparable to that obtained in a simultaneous compound in which both the CS1-US and CS2-US intervals were 800 msec, thus making CS2 less contiguous with the US than in the serial compound. Moreover, the level of responding to CS2 in all serial compound groups (blocking and control) was lower than in the simultaneous compound groups. The present findings provide further evidence that the associative consequences of CS-US contiguity can be highly attenuated by processes of attention or competition for associative strength.     

Effects of relative reinforcer frequency on complex color detection

Pigeons were trained under a discrete-trials detection procedure in which one of a set of color stimuli was presented on the center key and a single response turned off the stimulus and illuminated two side keys. Single responses to one or the other side key produced occasional reinforcers depending on the value of the color stimulus. In Experiment 1, one color-stimulus set comprised 559, 564, 569, and 574 nm, and right-key pecks were occasionally reinforced following presentations of members of this set. The other stimulus set comprised 579, 584, 589, and 594 nm, and left-key pecks were occasionally reinforced following presentations of members of this set. Across seven experimental conditions, the left/(left + right) relative reinforcer frequency was varied from .1 to .9. In Experiment 2, one stimulus set contained only one member, 574 nm, and right-key responses were occasionally reinforced following its presentation. Over 12 experimental conditions, two manipulations were carried out. First, the number of stimuli comprising the other stimulus set was increased from one (579 nm) to two (579 and 584 nm) to three (579, 584, and 589 nm) and to four (579, 584, 589, and 594 nm), and left-key responses were reinforced occasionally following center-key presentations of members of this set. Second, for each stimulus combination, the left/(left + right) relative reinforcer frequency was varied from .1 to .5 to .9 across three experimental conditions. The principal finding of Experiments 1 and 2 was that reinforcers and stimuli interacted in their effects on behavior. In Experiment 3, pairs of adjacent stimuli (5 nm apart) in the range 559 to 594 nm were presented in each experimental condition, and the left/(left + right) relative reinforcer frequency was held constant at .5. The data from all three experiments were analyzed according to a detection model describing performance in multiple-stimulus two-response procedures. This model provided independent measures of stimulus discriminability, contingency discriminability, and bias. The analysis showed that (a) consistent with the color-naming function, pigeons were better able to discriminate between higher nanometer values than lower nanometer values; (b) their ability to discriminate between the stimuli was independent of the number of wavelengths comprising each stimulus set; (c) they allocated delivered reinforcers very accurately to the previously emitted response; and (d) no consistent biases emerged.     

Facilitation of instrumental behavior by a Pavlovian appetitive conditioned stimulus

Three experiments examined appetitive Pavlovian-instrumental interactions by presenting separately trained conditioned stimuli (CSs) during reinforced instrumental responding in rabbits. Intra-oral reinforcement was used to minimize interference from peripheral responses such as magazine approach. In experiment 1, the rabbits were first trained to perform an instrumental head-raising response for sucrose reward. A conditioned jaw movement response was then established to a 2-sec CS by pairing it with sucrose; a control stimulus was unpaired with sucrose. Instrumental responding maintained by a variable-interval 40-sec schedule was enhanced during 10-sec presentations of the paired, but not the unpaired, CS. Responding on a variable-ratio 15 schedule was unaffected except on trials on which the pre-CS baseline response rate was low; in such cases the paired CS caused a long-lasting acceleration of responding. Noncontingent presentation of the sucrose reinforcer itself briefly suppressed responding but had no long-term effect. In Experiment 2, a CS that had been conditioned at a 10-sec duration produced the same pattern of effects as in the first study, indicating that facilitation resulted from CS presentation rather than from the frustrative effects of non-reinforcement of the CS. In Experiment 3 an inhibitory CS blocked facilitation by the excitatory CS but did not itself affect instrumental responding. These results support the view that Pavlovian processes play a positive role in instrumental performance and suggest that previous findings of suppression by a short-duration CS reflect peripheral interference. The dependence of facilitation on the baseline level of responding is discussed in terms of associative and motivational theories of Pavlovian mediation.     

The role of local context in autoshaping

Three experiments used an autoshaping procedure with pigeons to examine the effects of nonreinforced, nontarget stimuli (ITI-fillers) during the intertrial interval on responding to a reinforced target CS. Experiment 1 replicated a previous demonstration that an ITI-filler that occupied a substantial portion of the ITI attenuated responding to the target CS relative to a group trained with a similar ITI but lacking the ITI-filler. Experiment 2 found that the superiority of responding typically found with a long ITI relative to a short ITI, that is, the trial-spacing effect, can be reversed by imposing a filler stimulus during the ITI in the former condition. Experiment 3, using a within-subject design, found that when one background stimulus condition occupied a majority of the interreinforcement interval, pairings of one target CS with reinforcement that were embedded within this background condition, that is, a long duration local context occurred for this CS, yielded better performance that a second, reinforced, target CS paired with reinforcement in the shorter duration background condition. These results confirmed predictions derived from a local context view of cycle time. Comparator theories of classical conditioning require incorporation of this notion into their conceptualization of effective cycle time in order to explain the present findings.