Aspects of the reinforcer learned in second-order Pavlovian conditioning

Four experiments used an autoshaping procedure in pigeons to explore learning about the reinforcer in a second-order conditioning paradigm. Experiment 1 conditioned two visual second-order stimuli (S2), using as reinforcers two visual first-order stimuli (S1), each of which had previously been paired with food. Animals for which the S2 stimuli were each consistently paired with one particular S1 developed second-order responding more rapidly than did animals for which the identity of S1 varied from trial to trial. Moreover, following consistent pairings, extinction of an S1 had a depressive effect upon second-order responding which was peculiar to the S2 with which it had been paired. Both results suggest that in this preparation the organism identifies a particular S1 as the reinforcer for each S2. The remaining experiments examined the details of that identification. A compound S1, itself composed of two separable elements, was used to reinforce an S2. Subsequent extinction of either element of S1 led to a depression in the responding to S2, which indicates that both elements were involved in the second-order conditioning. Moreover, the use of several complex discriminations, which produced different behavior to S1 and to its elements, suggested that the organism had associated the S2 with the compound S1 rather than with its separate elements. However, even complete extinction of the response to S1 left some residual behavior to S2, which indicates that a portion of the second-order conditioning is independent of the current state of the reinforcer. These results demonstrate that in some situations the organism associates a conditioned stimulus with a rich representation of the reinforcer.     

Pavlovian second-order conditioned analgesia

Three experiments with rat subjects assessed conditioned analgesia in a Pavlovian second-order conditioning procedure by using inhibition of responding to thermal stimulation as an index of pain sensitivity. In Experiment 1, rats receiving second-order conditioning showed longer response latencies during a test of pain sensitivity in the presence of the second-order conditioned stimulus (CS) than rats receiving appropriate control procedures. Experiment 2 found that extinction of the first-order CS had no effect on established second-order conditioned analgesia. Experiment 3 evaluated the effects of post second-order conditioning pairings of morphine and the shock unconditioned stimulus (US). Rats receiving paired morphine-shock presentations showed significantly shorter response latencies during a hot-plate test of pain sensitivity in the presence of the second-order CS than did groups of rats receiving various control procedures; second-order analgesia was attenuated. These data extend the associative account of conditioned analgesia to second-order conditioning situations and are discussed in terms of the mediation of both first- and second-order analgesia by an association between the CS and a representation or expectancy of the US, which may directly activate endogenous pain inhibition systems.     

The representation of the reinforcer and the force of the pigeon's keypeck in first and second-order conditioning

The aim of this study was to determine whether reinforcer-specific conditioned responding would occur in a situation in which responding was not thought to be mediated by a representation that encodes information about the specific properties of the reinforcer. The force of the pigeon's keypeck was monitored during first and second-order conditioning with either food or water as the unconditioned stimulus (US). Each pigeon was trained with four different stimuli: a first-order cue predicting that responding would be reinforced with grain (S1f), a first-order cue predicting water as the reinforcer (S1w), a second-order stimulus predicting the S1f (S2f), and a second-order cue predicting the S1w (S2w). Following conditioning, the pigeons were selectively satiated with one of the two reinforcers and presented with the first and second-order cues in an extinction test. At the end of training, the pigeon's keypecks were less forceful to the S1w than to the S1f. There was not, however, a reliable difference between the force of the pecks to the S2f and the S2w. These force differences are consistent with the conclusion that the topography of the keypecks was systematically related to the nature of the primary reinforcer during first but not during second-order conditioning. The results from the selective satiation test are difficult to interpret. There was no evidence to indicate that second-order responding was mediated by a detailed representation of the primary US, but a detailed representation of the reinforcer may have been mediating first-order responding. Taken together, these findings are consistent with the view that a representation of the reinforcer is an important determinant of the topography of conditioned responding.     

Second-order schedules with paired auditory brief stimuli

Pigeons were exposed to multiple second-order schedules of paired and unpaired brief stimuli in which responding on the main key was reinforced according to a fixed-interval thirty-second schedule by a brief stimulus (a tone in the paired schedule) and advancement to the next segment of the second-order schedule. In Experiment 1, a response on the second key was required during the tone in its fourth and final presentation to produce food. Responses during earlier brief stimuli indicated the extent to which the final brief stimulus was discriminated from preceding ones. Responding was comparable during all tones, extending prior findings with visual paired brief stimuli and weakening explanations of subjects' failure to discriminate between brief-stimulus presentations in terms of elicited responding. In Experiment 2 the number of fixed-interval segments comprising the second-order schedules varied from one through eight. Although main-key response rates increased across segments in both experiments, they increased much less sharply with a variable number of segments. These results suggest that the increase in main-key response rates across segments is due primarily to a degree of temporal discrimination not reflected on the second key. Main-key response rates were higher on paired auditory brief-stimulus schedules than on unpaired visual brief-stimulus schedules, especially in Experiment 2, thus further extending findings with visual brief stimuli to second-order schedules with auditory brief stimuli.     

Differential effects of two ways of devaluing the unconditioned stimulus after Pavlovian appetitive conditioning

Three experiments with rat subjects investigated the effects of two methods of devaluing a food unconditioned stimulus (US) after pairings of an auditory conditioned stimulus (CS) with that US. Experiment 1 found no effect of postconditioning pairings of the food US with lithium chloride (LiCl) on general activity to a tone CS, even though those pairings substantially reduced food consumption. Experiments 2 and 3 compared the effects on conditioned responding of postconditioning pairings of food with LiCl and with high-speed rotation. In these experiments the general activity measure was supplemented by a detailed visual analysis of the rats' behavior. Experiment 2 found that food-rotation pairings had larger effects than food-LiCl pairings on general activity responding and on two detailed behavioral measures but that food-LiCl pairings had larger effects on food consumption and on one behavioral measure. Experiment 3 replicated the findings of Experiment 2 and found that the ability of the CS to serve as a reinforcer for second-order conditioning after US devaluation was reduced more by food-LiCl pairings.     

Representation-mediated extinction of conditioned flavor aversions

Conditioned flavor aversions were extinguished by presenting without consequence auditory stimuli that had been previously paired with the aversive flavor. In Experiment 1, rats that received tone-sucrose pairings, then sucrose-lithium chloride (LiCl) pairings, and finally repeated tone-alone presentations showed greater sucrose consumption in subsequent testing than rats that received similar sucrose-LiCl pairings and tone-alone presentations but no initial tone-sucrose pairings. Experiment 2 demonstrated the stimulus specificity of the mediated extinction observed in Experiment 1. In Experiment 3, rats that received first-order light-food and second-order tone-light pairings prior to sucrose-LiCl pairings did not show greater subsequent sucrose consumption when extinction of the second-order tone intervened. These results suggest that conditioned stimulus (CS)-evoked representations of events can substitute for those events themselves in the extinction of previously established associations.     

Reversibility of single-incentive selective associations.

Rats were trained to press a lever in the presence of a tone-light compound stimulus and not to press in its absence. In each of two experiments, schedules were designed to make the compound a conditioned punisher for one group and a conditioned reinforcer for the other. In Experiment 1, one group's responding produced food in the presence of the compound but not in its absence. The other group's responding terminated the compound stimulus, and food was presented only in its absence. When tone and light were later presented separately, light controlled more responding than did tone in the former group, but tone gained substantial control in the latter. The same effects were also observed within subjects when the training schedules were switched over groups. In Experiment 2, two groups avoided shock in the presence of the compound stimulus. In the absence of the compound, one group was not shocked, and the other received both response-independent and response-produced shock. When tone and light were presented separately, the former group's responding was mainly controlled by tone, but the latter group's responding was almost exclusively controlled by light. These effects were also observed within subjects when the training schedules were switched over groups. Thus, these single-incentive selective association effects (appetitive in Experiment 1 and aversive in Experiment 2) were completely reversible. The schedules in which the compound should have been a conditioned reinforcer consistently produced visual control, and auditory control increased when the compound should have become a conditioned punisher. Currently accepted accounts of selective associations based on affinities between shock and auditory stimuli and between food and visual stimuli (i.e., stimulus-reinforcer interactions) do not adequately address these results. The contingencies of reinforcement most recently associated with the compound and with its absence, rather than the nature of the reinforcer, determined whether auditory or visual stimulus control developed.     

Reinforcement magnitude effects in first- and second-order conditioning of directed action

Three experiments investigated the effects of reinforcement magnitude on conditioned key pecking in pigeons. Experiment 1, which included between-groups and within-subject designs, yielded significant effects of unconditioned stimulus (US) magnitude on the within-conditioned stimulus (CS) distribution of key pecks and on choice behavior, but no effect on the overall rate of key pecking. Experiment 2 employed a larger US-magnitude difference in a within-subject design. This manipulation resulted in differential rates of key pecking as well as a significant choice effect and differential within-CS key-peck distributions. A second-order conditioning procedure was used in Experiment 3, in which diffuse, visual stimuli (S1's) served as Pavlovian reinforcers for two key-light S2's. The S1 previously paired with a large US was more effective in conditioning second-order key-peck behavior to an S2 than was the S1 paired with a small US. The results of these experiments demonstrate that the associative effects of US magnitude can be expressed in the strength of CS-directed motor responding. The distinctive within-CS key-peck distributions in first-order conditioning suggests an interaction between CS- and US-directed responses.     

Summation: Further assessment of a configural theory

Rats received Pavlovian conditioning in which food was signalled by a visual stimulus, A+, an auditory stimulus, B+, and a compound composed of different visual and auditory stimuli, CD+. Test trials were then given with the compound AB. Experiments 1 and 2A revealed stronger responding during AB than during CD. In Experiment 2B, there was no evidence of a summation of responding during AB when A+ B+ training was conducted in the absence of CD+ trials. A further failure to observe abnormally strong responding during ABwas found in Experiment 3 for which the training trials with A+ B+ CD+ were accompanied by trials in which C and D were separately paired with food. The results are explained in terms of a configural theory of conditioning, which assumes that responding during a compound is determined by generalization from its components, as well as from other compounds to which it is similar.     

The effects of satiation after first— and second-order appetitive conditioning in rats

Rats received first- and second-order conditioning based on a food unconditional stimulus (US). The effects of postconditioning satiation on the performance of detailed behavioral components of general activity evoked by first- and second-order conditional stimuli (CSs) were then assessed in extinction tests. Satiation reduced the frequency of all components of general activity evoked by first-order CSs but had little or no effects on behaviors evoked by second-order CSs. These results are consistent with earlier suggestions that first-but not second-order conditional responding is mediated by a representation of the US and that a major effect of satiation is to devalue that representation.     

Spatial contiguity facilitates Pavlovian second-order conditioning

Two experiments investigated the effects of spatial contiguity upon the formation of second-order conditioning in pigeon subjects. Experiment 1 used an autoshaping procedure to pair two visual stimuli, S2 and S1, after S1 had previously been paired with food. The resulting second-order conditioning of S2 was superior when both stimuli appeared on the same response key within a trial, compared with their appearing on different keys. Experiment 2 found a similar importance of spatial contiguity between S2 and S1 in a conditioned suppression paradigm. In addition, consistently presenting S2 and S1 in the same spatial location produced superior conditioning compared with varying their spatial relation from trial to trial. The design of these experiments was such as to imply that spatial contiguity facilitates performance by improving the formation of association rather than by promoting stimulus generalization or pseudoconditioning. Moreover, the observation of a facilitative effect of spatial contiguity between S1 and S2 in two different paradigms that use qualitatively different unconditioned stimuli and evoke different responses implies some generality for these findings. Consequently, these results suggest that spatially contiguous stimuli are especially associable in Pavlovian conditioning paradigms.     

Second-order autoshaped key pecking based on an auditory stimulus.

In Experiment 1, pigeons were exposed either to paired or to unpaired presentations of a tone and grain, and then to paired presentations of a keylight with the tone. Substantial second-order conditioned pecking to the keylight was produced in the birds that had received paired presentations of tone and grain. In Experiment 2, second-order pecking to the keylight increased in probability across four groups that had received, respectively, 20, 80, 140, or 200 paired presentations of tone and grain. In Experiment 3, the amount of pecking directed towards a keylight which predicted the first-order, tone CS was as substantial in birds without a prior history of key pecking as in birds with such a history. A further experiment failed to discover any significant differences in the levels of second-order pecking to a keylight paired with a first-order tone CS or with a first-order keylight CS. Thus, an auditory signal that does not itself support pecking may enable a localized visual stimulus to evoke key pecking.     

An analysis of second-order autoshaping

Three mechanisms can explain second-order conditioning: (1) The second-order conditioned stimulus (CS2) could activate a representation of the first-order conditioned stimulus (CS1), thereby provoking the conditioned response (CR); The CS2 could enter into an excitatory association with either (2) the representation governing the CR, or (3) with a representation of the reinforcer evoked by the CS1. A series of experiments using second-order autoshaping with birds was performed to examine these possibilities. Following second-order autoshaping, birds’ responding to the CS2 was found to be unaffected by extinction of the CS1, a result interpreted as showing mechanism (1) to be unimportant and implicating either one or both of the other mechanisms. The CS2 was also shown to provide the birds with specific sensory information about the reinforcer, a pattern of results uniquely predicted by mechanism (3). Implications for the understanding of second-order conditioning and for the SOP model (Wagner, 1981) of associative learning are discussed.     

Conditioned Inhibition Produced by Extinction of a Conditioned Stimulus

Four experiments used a conditioned taste aversion procedure to examine the potential for CS-alone extinction treatment to produce a conditioned stimulus that possesses inhibitory properties. In Experiment 1, saccharin was paired with LiCl, and then saccharin was presented alone for several trials to produce extensive behavioral extinction. Animals receiving this treatment were retarded in reacquiring conditioned responding to saccharin relative to control subjects receiving conditioning to the flavor for the first time. In Experiment 2, the extinguished saccharin stimulus was shown to decrease conditioned responding to a known excitor when the two stimuli were presented in compound as a summation test. Experiments 3A and 3B replicated the findings of Experiments 1 and 2 while providing evidence that the effects were not due to the differential effects of neophobia during testing. These three experiments revealed that an extinguished conditioned excitor passes retardation and summation tests for conditioned inhibition. Experiment 4 found that extinction of a known excitor was slowed when the excitor was extinguished in compound with a previously extinguished conditioned stimulus. That is, an extinguished CS provided protection from extinction to another CS, a finding also consistent with the view that extinction produces conditioned inhibition.     

Higher order autoshaping

A series of experiments is reported on appetitive higher order conditioning in the pigeon. Experiment I showed that second order autoshaping can be produced by pairing a neutral keylight with a keylight of another colour, previously paired with food. Experiment II employed an omission procedure to show that second order autoshaping is a consequence of the contingency between first and second order stimuli. In Experiment III, extinction of responding to the first order stimulus was shown to reduce responding to the second order stimulus. Experiments IV and V showed firstly that this reduction is not due to generalization of extinction, and secondly that second order key pecks may be produced in the absence of any pecking to the first order stimulus. The results suggest that second order autoshaping is based largely on a direct association between the first and second order stimuli.     

Effects of contextual conditioning and unconditional stimulus presentation on performance in appetitive conditioning

Four experiments with rat subjects examined the effects of contextual conditioning on conditioned appetitive performance. Experiment 1 compared the effects of contextual conditioning on performance to conditioned stimuli (CSs) with different conditioning histories. Contextual conditioning enhanced performance to the CS if the CS had first been conditioned and then extinguished, but had no effect on performance when the CS had been merely paired or unpaired with food. Experiments 2 and 3 then asked whether the effect on the extinguished CS was due to contextual conditioning acting as a cue for conditioning. In Experiment 2, extinction procedures in which extra unconditioned stimuli (USs) were presented during the intertrial intervals were found to reduce the CS's sensitivity to enhancement by contextual conditioning, but had no effect on spontaneous recovery. In Experiment 3, USs added to conditioning or extinction acquired the ability to cue the corresponding performance. Under some conditions, USs added to conditioning could suppress performance (Experiment 4). The results suggest that contextual conditioning has complex effects that can be better understood by recognizing that contextual conditioning, as well as the USs that create it,Mayacquire discriminative control over conditioned responding.     

Stimulus control of schedule-induced activity in pigeons during multiple schedules

Stimulus control of schedule-induced general activity was demonstrated with pigeons using multiple schedules of response-independent food delivery. In Experiment 1, the introduction of food during a multiple variable-time 30-second variable-time 30-second schedule produced a tenfold increase in activity above the no-food baseline. Each pigeon developed stable differential activity rates during the components (correlated with red and green lights) of a multiple variable-time 30-second extinction schedule. Lengthening the extinction component from 1 to 7 minutes increased the rate differences and produced a reliable pattern of responding during S− (the stimulus correlated with extinction): Activity rate was high immediately following the change from S+ (the stimulus correlated with variable-time 30-second) to S−, then decreased abruptly and remained low throughout the middle of the interval, and subsequently showed a positively accelerated increase until the stimulus changed to S+. In Experiment 2, three pigeons were exposed to a mixed variable-time extinction schedule prior to a multiple variable-time extinction schedule. Auditory rather than visual stimuli were used to determine the generality of Experiment 1 results. The multiple- versus mixed-schedule results indicated that stimulus control of activity occurred for two of the birds, but rate differences between S+ and S− were much less than those demonstrated with visual stimuli. A direct comparison of visual and auditory stimulus control in Experiment 3 supported this conclusion. These parallels between the stimulus control of reinforced responding and that of schedule-induced activity suggest that the stimulus control of induced activity may be a factor in operant stimulus control.     

The transfer of respondent eliciting and extinction functions through stimulus equivalence classes

Two studies investigated the transfer of respondent elicitation through equivalence classes. In Experiment 1, match-to-sample procedures were used to teach 8 subjects two four-member equivalence classes. One member of one class was then paired with electric shock, and one member of the other class was presented without shock. All remaining stimuli were then presented. Using skin conductance as the measure of conditioning, transfer of conditioning was demonstrated in 6 of the 8 subjects. In Experiment 2, similar procedures were used to replicate the results of Experiment 1 and investigate the transfer of extinction. Following equivalence training and conditioning to all members of one class, one member was then presented in extinction. When the remaining stimuli from this class were then presented, they failed to elicit skin conductance. In the final phase of the experiment, the stimulus that was previously presented in extinction was reconditioned. Test trials with other members of the class revealed that they regained elicitation function. These results demonstrate that both respondent elicitation and extinction can transfer through stimulus classes. The clinical and applied significance of the results is discussed.     

Mechanisms of second-order conditioning with a backward conditioned stimulus

Five conditioned suppression experiments with rats examined the conditions under which backward pairings endow a first-order conditioned stimulus (CS1) with the ability to serve as a secondary reinforcer. Experiments 2-5B found evidence for excitatory second-order conditioning (SOC) if, during first-order pairings, the US-CS1 interval was 0 s rather than 3 s. Levels of SOC were comparable after forward and backward pairings (Experiments 1-3), and were unaffected by extinction of CS1 after SOC (Experiment 3). These results suggest that forward and backward CS1s support SOC for the same reason, and they call into question the need to invoke any special mechanism such as memory integration.     

A comparison of the effects of partial reinforcement schedules using a within-subject serial autoshaping procedure

Three experiments, each using a single group of pigeons, are reported. In Experiment 1 subjects were initially trained with two stimuli, one of which was always followed by food, the other being reinforced according to a 50% partial reinforcement schedule. Subsequently a serial procedure was adopted in which an additional stimulus, C, was consistently followed by the partially reinforced CS. A second additional stimulus, A, was followed on half of its occurrences by the continuously reinforced CS, its remaining presentations being followed by nothing. The rate of autoshaped keypecking was substantially greater during A than during C. In the remaining experiments subjects received first-order conditioning with a single stimulus that was either partially (Experiment 2) or continuously (Experiment 3) reinforced. The stimuli A and C were then again introduced for serial autoshaping. Stimulus A was occasionally paired with the CS and occasionally followed by nothing, whereas stimulus C was always followed by the CS. As in Experiment 1, the rate of responding during A was greater than during C. It is proposed that one influence on the rate of autoshaped keypecking during a CS is the accuracy with which the immediate consequences of that CS are predicted.