The comparator hypothesis of conditioned response generation: manifest conditioned excitation and inhibition as a function of relative excitatory strengths of CS and conditioning context at the time of testing

In the present research water-deprived rats were used in a conditioned lick suppression paradigm to test and further develop Rescorla's (1968) contingency theory, which posits that excitatory associations are formed when a conditioned stimulus (CS) signals an increase in unconditioned stimulus (US) likelihood and that inhibitory associations develop when the CS signals a decrease in US likelihood. In Experiment 1 we found that responding to a CS varied inversely with the associative status of the context in which the CS was trained and that this response was unaltered when testing occurred in a distinctively dissimilar context with a different conditioning history, provided associative summation with the test context was minimized. These results suggest that manifest excitatory and inhibitory conditioned responding is modulated by the associative value of the training context rather than that of the test context. In Experiment 2 it was demonstrated that postconditioning decreases in the associative value of the CS training context reduced the effective inhibitory value of the CS even when testing occurred outside of the training context. Moreover, this contextual deflation effect was specific to the CS training context as opposed to any other excitatory context. Collectively, these studies support the comparator hypothesis, which states that conditioned responding is determined by a comparison of the associative strengths of the CS and its training context that occurs at the time of testing rather than at the time of conditioning. This implies that all associations are excitatory and that responding indicative of conditioned inhibition reflects a CS-US association that is below (or near) the associative strength of its comparator stimulus. It is suggested that response rules which go beyond a monotonic relation between associative value and response strength can partially relieve learning theories of their explanatory burdens, thereby allowing for simpler models of acquisition.     

Response rates track the history of reinforcement times

When conditioning involves a consistent temporal relationship between the conditioned stimulus (CS) and unconditioned stimulus (US), the expression of conditioned responses within a trial peaks at the usual time of the US relative to the CS. Here we examine the temporal profile of responses during conditioning with variable CS-US intervals. We conditioned stimuli with either uniformly distributed or exponentially distributed random CS-US intervals. In the former case, the frequency of each CS-US interval within a specified range is uniform but the momentary probability of the US (the hazard function) increases as time elapses during the trial; with the latter distribution, short CS-US intervals are more frequent than longer intervals, but the momentary probability of the US is constant across time within the trial. We report that, in a magazine approach paradigm, rats' response rates remained stable as time elapses during the CS when the CS-US intervals were uniformly distributed, whereas their response rates declined when the CS-US intervals were exponentially distributed. In other words, the profile of responding during the CS matched the frequency distribution of the US times, not the momentary probability of the US during the CS. These results are inconsistent with real-time associative models, which predict that associative strength tracks the momentary probability of the US, but may provide support for timing models of conditioning in which conditioned responding is tied to remembered times of reinforcement.     

Blocking but not conditioned inhibition results when an added stimulus is reinforced in compound with multiple pretrained stimuli

Conditioned barpress suppression by rats was used to explore the associative status of an initially neutral stimulus that was reinforced in simultaneous compound with two independently pretrained conditioned excitors. In contrast to the Rescorla-Wagner model, which predicts that the target stimulus (X) will be inhibitory following such A+/B+/ABX+ training, the present study found no evidence that X acquired inhibitory associative strength. Rather, the pretrained stimuli merely served to block conditioned excitatory responding to the target stimulus.     

Contrast effects in maintained generalization gradients

In Experiment I, pigeons were given equal reinforcement (variable-interval 1-min) for responding during randomized presentations of eight line-orientation stimuli. Then, only responding in the vertical orientation was reinforced. Stable generalization gradients soon formed and persistent behavioral and local (transient) contrast effects appeared. Local contrast effects were not a function of relative reinforcement frequency or of any other variable known to produce contrast. Instead, they were related to average response rates associated with each stimulus. Experiment II showed that local contrast effects represent increases and decreases in response rates relative to baseline responding, and that these effects are relative; a given stimulus might enhance responding during a subsequent presentation of one stimulus, but depress responding when followed by another. These data indicate that discrimination learning is not adequately described as the acquisition of excitatory properties by some stimuli and inhibitory properties by others. A more adequate account implies that stimuli exert both excitatory and inhibitory effects related to their value.     

Conditioning of the rabbit's nictitating membrane response to a CSA-CSB-US serial compound: manipulations of CSB's associative character.

Response acquisition to a trace conditioned stimulus (CSA) can be facilitated by insertion of a second stimulus (CSB) at the end of the trace interval just before the unconditioned stimulus (US). This effect may arise from serial mediation of trace conditioning, second-order conditioning, or both. Whereas serial mediation relies only on the presence of CSB, associative transfer relies on CSB's associative strength. In the present experiments, the presence of CSB was fixed, whereas CSB's associative strength was manipulated by (a) extinction of CSB, (b) latent inhibition of CSB, and (c) prior CSB-US pairings. In the first 2 cases, the level of responding to CSA was reduced in a fashion parallel to that of CSB. However, in the third case, partial blocking of conditioned response (CR) acquisition to CSA was observed. The results are discussed with reference to the role of associative transfer to both facilitating and blocking CR acquisition to CSA.     

The interaction between discriminative stimuli and outcomes during instrumental learning

Rats were trained on a biconditional discrimination in which the delivery of a food pellet stimulus signalled that pressing on one of two levers would be reinforced, whereas the delivery of a sucrose solution stimulus signalled that the reward was contingent on pressing the other lever. The outcome was the same food type as the discriminative stimulus in the congruent group but the other food type in the incongruent group. Both responses were rewarded with the same outcome in the same group. All the three groups learned the discrimination at statistically indistinguishable rates. Prefeeding one of the outcomes selectively reduced the associated response thereby demonstrating that responding was mediated by a representation of the outcome. Moreover, the outcome of one trial controlled responding on the next trial in accord with the stimulus function of the food type. These results are discussed in relation to the associative structures mediating the discriminative control of instrumental performance.     

Associative changes in excitors and inhibitors differ when they are conditioned in compound

When an AB stimulus compound is reinforced or nonreinforced, there are associative changes in both A and B elements. In many contemporary theories those changes are viewed as governed by a common error term, computed as the discrepancy between the total associative strength of the AB compound and that supported by the trial consequence. This implies that if A and B are equally salient, then the magnitude of their associative change should be the same, whatever their strengths prior to the AB trial. This implication was explored for a compound consisting of an excitatory A and an inhibitory B. A novel assessment procedure avoided the difficulty of making comparisons at different locations on the performance scale. Three experiments using a magazine approach preparation in rats and 3 using autoshaping in pigeons found evidence contradicting this implication. The excitatory A changed less than the inhibitory B when the compound was reinforced but more than B when the compound was nonreinforced.     

Timing at the Start of Associative Learning

Some theories of associative learning imply that time plays a fundamental role in the acquisition process. Consistent with these theories, this paper presents evidence that the time from the onset of a conditioned stimulus (CS) until presentation of the unconditioned stimulus (US) is learned very rapidly at the start of training. We report two autoshaping studies and a study on aversive conditioning in goldfish in which we examine timing at the start of conditioning. We also review data from a number of other conditioning preparations, including fear-potentiated startle, appetitive conditioning in rats, and eyeblink conditioning in rabbits, that report conditioned response (CR) timing early in training. Acquisition speed and the very first expressions of conditioned responding often show sensitivity to the time of US presentation. In instances where temporal control is slowly expressed, it is likely due to performance factors, not to slow learning about time. In fact, the learning about time may be a necessary condition for associative learning.     

Are associative changes in acquisition and extinction negatively accelerated?

Performance to a conditioned stimulus typically shows a negatively accelerated change with Pavlovian conditioning or extinction. This is commonly attributed to negatively accelerated changes in the underlying associative learning. However, the absence of well-specified rules for mapping learning into performance makes it difficult to infer changes in the size of associative changes for stimuli occupying different points on the performance scale. This problem can be addressed by comparing responding to stimulus compounds each consisting of 2 stimuli, 1 of which is poorly conditioned and 1 of which is well conditioned, when either the former or the latter receives a specified amount of additional training. This permits comparison at a common performance level. Two magazine-approach experiments in rats and 2 autoshaping experiments in pigeons used this technique to assess the form of the associative changes in acquisition and extinction.     

Responding to a conditioned stimulus depends on the current associative status of other cues present during training of that specific stimulus.

The comparator hypothesis is a response rule stating that responding to a Pavlovian conditioned stimulus (CS) reflects the associative strength of the CS relative to that of other cues (comparator stimuli) that were present during CS training. Thus, modulation of the associative strength of a CS's comparator stimulus should alter responding to that CS. These studies examined the stimulus specificity of this effect using within-subjects designs. Rats were trained on 2 CSs, each with a unique comparator stimulus, to determine the degree to which posttraining extinction of the comparator stimulus for one CS influences responding to the other CS. Using negative contingency (Experiments 1 and 2), overshadowing (Experiment 3), and local context (Experiment 4) preparations, stimulus specificity was observed. In each case, posttraining extinction of the comparator stimulus for one CS had greater impact on responding to that CS than on responding to the alternate CS.     

Trial spacing is a determinant of cue interaction

Four conditioned lick suppression experiments with rats examined the effect of trial spacing on cue interaction. Experiments 1 and 2 found overshadowing to be eliminated with massed compound stimulus-outcome pairings and the usual trial spacing effect to be reversed with compound acquisition trials. Experiment 3 found that whether acquisition compound-outcome pairings were massed or spaced determined the effect of posttraining extinction treatment. Extinction of the overshadowing cue reduced responding following massed training and increased responding following spaced training. Extinction of the context decreased responding following massed training. Experiment 4 found the conditioning and devaluation results to be associative and stimulus specific. These results are in accord with the extended comparator hypothesis (J. C. Denniston, H. I. Savastano, & R. R. Miller, 2001).     

Savings tests: separating differences in rate of learning from differences in initial levels

Three experiments used a Pavlovian magazine-approach procedure in rats to explore, in 3 theoretically interesting cases, the inferences made from savings tests. In each experiment, a compound test procedure allowed the separation of differences in rate of acquiring new associative learning from differences in initial level of associative strength. Experiment 1 found that the slower acquisition after prior nonreinforced exposure (latent inhibition) reflected differences in learning rate, not initial level. By contrast, Experiments 2 and 3 found that the higher performance observed to a previously trained and extinguished stimulus, and lower performance observed to a conditioned inhibitor, both arose primarily because of differences in initial associative value. These experiments illustrate the usefulness of a novel testing procedure in sharpening the conclusions from savings procedures.     

A response rule for positive and negative stimulus interaction in associative learning and performance

The present article presents a response rule developed to account for both positive and negative stimulus interaction. In the response rule proposed here, positive interaction phenomena (e.g., second-order conditioning) and negative interaction phenomena (e.g., Pavlovian conditioned inhibition) are presumed to occur during performance and acquisition, respectively. Also, in this rule the novelty of the test stimulus determines the expression of positive interaction on responding. As the stimulus loses its novelty over training, positive interaction effects will wane, which will allow negative interaction effects to emerge in responding elicited by the stimulus. It is proposed that this response rule can be adopted by acquisition-focused associative models (e.g., Rescorla & Wagner, 1972) in order to account for contrary associative phenomena in the literature. The simulation program used in this study is available for download from the author’s website and from the Psychonomic Society website at www.psychonomic.org/archive.     

Sign-tracking behavior in aversive conditioning: Its acquisition via a Pavlovian mechanism and its suppression by operant contingencies

When rats are presented with Pavlovian backward pairings in which shock is the unconditioned stimulus (US) and an elevated platform the conditioned stimulus (CS), they show a strong tendency to approach the platform. Attempts were made at evaluating the nature of the associative mechanisms responsible for the acwuisition and maintenance of the approach behavior. In Experiment 1, following a baseline period when platform presentations were given in the absence of shocks, first (training I), approach to the platform was prevented and subjects were exposed to either backward, forward, or random presentations of shock and platform: this procedure attempted to minimize the effects of response learning on the acquisition of approach behavior. Then (training II), all subjects received backward shock-platform pairings and had access to the platform. Approach to the platform was stronger in animals previously exposed to backward pairings than in the animals of the other two groups. Also, animals preexposed to forward platform-shock pairings showed suppression of approach behavior on the first trial of training II relative to their performance on the last trial of baseline. These data suggest that a stimulus-reinforcer learning mechanism alone is sufficent for the acquisition of approach. Experiments 2a and 2b evaluated the effects of “punishing” operant contingencies on Pavlovian-produced approach behavior. The results showed that approach can be eliminated, but only after repeated exposure to an operant contingency which consisted of the immediate closure of the platform upon approach responding. These data suggest that approach behavior produced by Pavlovian contingencies can be modified by operant contingencies. Taken together, these data provide additional support for a sign-tracking notion in aversive conditioning.     

The fate of redundant cues during blocking and a simple discrimination

In each of three experiments animals received blocking, A+ AX+, in which food was always presented after one stimulus, A, that was occasionally accompanied by another stimulus, X. They also received a simple discrimination, AX+ BX-, in which the presence and absence of food was signaled by two compounds that contained one unique cue, A or B, and one common cue, X. In each of these designs, X can be said to be redundant relative to A as a signal for food. Test trials at the end of training revealed that responding during X was stronger after blocking than after the simple discrimination. These results contradict predictions from theories of learning that assume changes in associative strength of a stimulus are determined by a global error term based on the outcome predicted by all the stimuli that are present for a conditioning trial. The results are interpreted, instead, by assuming either that animals store a memory of every trial to which they have been exposed, or that learning is governed by an error term based on the significance of individual stimuli.     

The content of compound conditioning

In three experiments using Pavlovian conditioning of magazine approach, rats were trained with a compound stimulus, AB, and were concurrently trained with stimulus B on its own. The reinforcement rate of B, rB, was either 1/2, 2/3, or 2/5 of rAB. After extended training, the conditioning strength of A was assessed using probe trials in which A was presented alone. Responding during A was compared with that during AB, B, and a third stimulus, C, for which rC = rAB - rB. In each experiment, the rats' response rate during A was almost identical to that during C (and during B, when rB = 1/2rAB). This suggests that, during AB conditioning, the rats had learned about rA as being equal to [rAB - rB], and implies that the content of their learning was a linear function of r. The findings provide strong support for rate-based models of conditioning (e.g., Gallistel & Gibbon, 2000). They are also consistent with the associative account of learning defined in the Rescorla and Wagner (1972) model, but only if the learning rate during reinforcement equals that during nonreinforcement.     

The effects of genotype, foraging role, and sucrose responsiveness on the tactile learning performance of honey bees (Apis mellifera L.)

We analyzed sucrose responsiveness and associative tactile learning in two genetic strains of honey bees under laboratory conditions. These strains differ in their foraging behavior. Bees of the "high" strain preferentially collect pollen. "Low"-strain bees mainly forage for nectar. Responsiveness to different sucrose concentrations and tactile learning were examined using the proboscis extension reflex. Acquisition, extinction of conditioned responses, and responses to an alternative tactile stimulus were tested. High-strain bees are more responsive to sucrose than low-strain bees. Regardless of genotype, pollen foragers are more responsive to sucrose than nectar foragers. In bees of both strains we find the same relationship between responsiveness to sucrose and acquisition. Bees responding to low sucrose concentrations show more often the conditioned response during acquisition than those responding only to higher sucrose concentrations. Extinction of conditioned responses depends on the response probability during acquisition. Discrimination between the two tactile stimuli is affected by genotype but not by responsiveness to sucrose. High-strain bees discriminate better than low-strain bees. Our experiments thus establish links between division of labor, responsiveness to sucrose, and associative learning in honey bees.     

The effect of deprivation level during noncontingent pairings and instrumental learning on subsequent instrumental performance

In Experiment 1, rats received either response-noncontingent pairings of a tone stimulus with food or response-contingent instrumental training during which only responses in the presence of the tone were reinforced. During this training rats were maintained at either 70 or 90% of their growth-adjusted predeprivation body weights. In a subsequent test phase, one-half of the subjects in each training condition were tested under either 70 or 90% body weight. Subjects which had received response-contingent instrumental training under 70% body weight responded significantly faster to presentations of the tone than did 90% body weight-trained subjects, regardless of test deprivation condition. There was no effect of the deprivation level in effect during response-noncontingent pairings. In Experiment 2, rats received noncontingent tone-reward pairings or nonpairings under either 70 or 90% body weight. In a later test phase under 90% body weight, instrumental responding to the tone was significantly faster for subjects which had received the tone-reward pairings. Deprivation level during the pairings again produced no effect. These results support two conclusions. First, the expectancy learning process appears to be relatively independent of deprivation conditions in effect when such learning takes place; and secondly, deprivation conditions in effect during instrumental learning affect the strength of S-R associative bonds formed.     

Blocking and unblocking of conditioned analgesia

Two experiments with rat subjects assessed the blocking phenomenon using inhibition of responding to painful thermal stimulation as an index of conditioned analgesia established through conditioned stimulus (CS)-shock pairings. In Experiment 1, a group of rats receiving the standard blocking procedure ($\text{A+}\text{AB+}$) showed shorter response latencies during a hot plate test of pain sensitivity in the presence of the added CS than groups of rats receiving various blocking control procedures. Experiment 2 replicated the blocking outcome of Experiment 1 and also demonstrated that the addition of a second shock unconditioned stimulus (US) during the compound conditioning phase attenuated the blocking effect; i.e., unblocking was observed. However, the deletion of a second shock US during compound conditioning in a group that had received two shocks on each trial in the first phase of the experiment failed to result in unblocking. These data extend the associative account of conditioned analgesia to situations involving stimulus selection and the implications for the analysis of aversive learning are discussed.