Second-order conditioning in Drosophila

Associative conditioning in Drosophila melanogaster has been well documented for several decades. However, most studies report only simple associations of conditioned stimuli (CS, e.g., odor) with unconditioned stimuli (US, e.g., electric shock) to measure learning or establish memory. Here we describe a straightforward second-order conditioning (SOC) protocol that further demonstrates the flexibility of fly behavior. In SOC, a previously conditioned stimulus (CS1) is used as reinforcement for a second conditioned stimulus (CS2) in associative learning. This higher-order context presents an opportunity for reassessing the roles of known learning and memory genes and neuronal networks in a new behavioral paradigm.     

Hippocampal associative cellular responses: dissociation with behavioral responses revealed by a transfer-of-control technique

Multiunit activity was recorded in the CA3 field of the dorsal hippocampus in freely moving rats during classical conditioning and subsequent presentation of the CS on operant baselines for food reward as well as shock avoidance. Rats were first trained in a nonsignaled bar-pressing-dependent shock omission task and in a food-motivated lever-pressing task (60-s VI). Five sessions with presentations of a previously habituated tone as a CS paired with footshock as a US were then given. Testing was carried out by presenting the CS alone while behavioral responses were maintained by reinforcement in both instrumental tasks on alternate sessions. As expected, the CS induced a marked suppression of lever pressing for food reward and a marked enhancement of bar-pressing for shock avoidance. The analysis of the frequency of multiunit discharges to the CS revealed that the hippocampal cellular responses established during classical conditioning were maintained while two different behavioral responses were exhibited to the CS. The results showed that the associative response of hippocampal neurons may be dissociated from the Pavlovian conditioned responses the CS elicits. They support the hypothesis that hippocampal cellular responses represent a neural index of the acquired CS-US associative representation.     

Pavlovian conditioning in human skilled motor behavior

The role of Pavlovian contingencies in human skilled motor behavior was investigated in three experiments by means of a new conditioning preparation. In Experiment 1 the present method was shown to be appropriate for the study of associative learning. Subjects who experienced a standard delay configuration performed significantly more conditioned responses than subjects who received either backward conditioning or random pairings. Stimulus generalization was shown to be slight in two additional groups. Subsequent experiments examined conditioning with multiple conditioned stimuli (CSs). In particular, in Experiment 2 some reciprocal overshadowing was demonstrated when two conditional stimuli (tone and vibration) were compounded. Experiment 3 investigated blocking. Blocking was less than expected, however. Subjects’ perceptions of the stimuli and reaction time data suggest that a certain proportion had shifted their attention to the added element of the CS compound. Results are discussed in relation to other studies on Pavlovian learning in humans and animals, which are concerned with “stimulus selection.”     

Simultaneous backward conditioned inhibition and mediated conditioning

Demonstrations of retrospective revaluation suggest that remembered stimuli undergo a reduction in association with the unconditioned stimulus (US) present during learning. Conversely, demonstrations of mediated conditioning in flavor-conditioning experiments with rats suggest that remembered stimuli undergo an increase in association with the US present during learning. In a food allergy prediction task with 23 undergraduates, we demonstrated simultaneous backward conditioned inhibition and mediated conditioning effects. These results are compatible with the hypothesis that the direction of change (decrease or increase) in associative strength depends on whether the remembered stimulus was of a different category (conditioned stimulus/antecedent) or the same category (US/outcome) as the presented US.     

The comparator hypothesis of conditioned response generation: manifest conditioned excitation and inhibition as a function of relative excitatory strengths of CS and conditioning context at the time of testing

In the present research water-deprived rats were used in a conditioned lick suppression paradigm to test and further develop Rescorla's (1968) contingency theory, which posits that excitatory associations are formed when a conditioned stimulus (CS) signals an increase in unconditioned stimulus (US) likelihood and that inhibitory associations develop when the CS signals a decrease in US likelihood. In Experiment 1 we found that responding to a CS varied inversely with the associative status of the context in which the CS was trained and that this response was unaltered when testing occurred in a distinctively dissimilar context with a different conditioning history, provided associative summation with the test context was minimized. These results suggest that manifest excitatory and inhibitory conditioned responding is modulated by the associative value of the training context rather than that of the test context. In Experiment 2 it was demonstrated that postconditioning decreases in the associative value of the CS training context reduced the effective inhibitory value of the CS even when testing occurred outside of the training context. Moreover, this contextual deflation effect was specific to the CS training context as opposed to any other excitatory context. Collectively, these studies support the comparator hypothesis, which states that conditioned responding is determined by a comparison of the associative strengths of the CS and its training context that occurs at the time of testing rather than at the time of conditioning. This implies that all associations are excitatory and that responding indicative of conditioned inhibition reflects a CS-US association that is below (or near) the associative strength of its comparator stimulus. It is suggested that response rules which go beyond a monotonic relation between associative value and response strength can partially relieve learning theories of their explanatory burdens, thereby allowing for simpler models of acquisition.     

Relative effectiveness of concurrent forward/backward versus simple forward and simple backward Pavlovian conditioning procedures

Concurrent forward/backward Pavlovian conditioning procedures were compared with simple forward and simple backward procedures for differential excitatory and inhibitory conditioning. Four groups of dogs each received 30 shocks while restrained in a Pavlovian harness. One group of dogs (CON) received concurrent conditioning with a clicker signaling shocks and with a tone following shocks. For another group of dogs (SF), a clicker signaled the shocks, whereas the tone was randomly related to shocks. For a third group (SB), the tone followed shock termination, whereas the clicker was randomly related to shocks. For a fourth group (TRC), both the clicker and the tone were randomly related to shocks. All stimuli were then superimposed on a temporally paced avoidance baseline and their associative properties were assessed relative to the TRC group. Differential excitatory-inhibitory conditioning was significantly reduced in the concurrent procedure relative to that which occurred in the simple conditioning procedures despite exactly equal CS + -US and US-CS-contingencies. These results were interpreted with reference to contextual conditioning effects predicted by the Rescorla-Wagner conditioning model and the theoretical issue of independence vs. interdependence of CS-US associative strengths.     

Associative learning acquisition and retention depends on developmental stage in Lymnaea stagnalis

Associative learning dependent on visual and vestibular sensory neurons and the underlying cellular mechanisms have been well characterized in Hermissenda but not yet in Lymnaea. Three days of conditioning with paired presentations of a light flash (conditional stimulus: CS) and orbital rotation (unconditional stimulus: UCS) in intact Lymnaea stagnalis results in a whole-body withdrawal response (WBWR) to the CS. In the current study, we examined the optimal stimulus conditions for associative learning, including developmental stage, number of stimuli, interstimulus interval, and intertrial interval. Animals with a shell length longer than 18 mm (sexually mature) acquired and retained the associative memory, while younger ones having a shell length shorter than 15 mm acquired but did not retain the memory to the following day. For mature animals, 10 paired presentations of the CS and UCS presented every 2 min were sufficient for the induction of a WBWR to the CS. Furthermore, animals conditioned with the UCS presented simultaneously with the last 2 s of the CS also exhibited a significant WBWR in response to the CS. Blind animals did not acquire the associative memory, suggesting that ocular photoreceptors, and not dermal photoreceptors, detected the CS. These results show that maturity was key to retention of associative learning.     

Feeding behavior of Aplysia: a model system for comparing cellular mechanisms of classical and operant conditioning

Feeding behavior of Aplysia provides an excellent model system for analyzing and comparing mechanisms underlying appetitive classical conditioning and reward operant conditioning. Behavioral protocols have been developed for both forms of associative learning, both of which increase the occurrence of biting following training. Because the neural circuitry that mediates the behavior is well characterized and amenable to detailed cellular analyses, substantial progress has been made toward a comparative analysis of the cellular mechanisms underlying these two forms of associative learning. Both forms of associative learning use the same reinforcement pathway (the esophageal nerve, En) and the same reinforcement transmitter (dopamine, DA). In addition, at least one cellular locus of plasticity (cell B51) is modified by both forms of associative learning. However, the two forms of associative learning have opposite effects on B51. Classical conditioning decreases the excitability of B51, whereas operant conditioning increases the excitability of B51. Thus, the approach of using two forms of associative learning to modify a single behavior, which is mediated by an analytically tractable neural circuit, is revealing similarities and differences in the mechanisms that underlie classical and operant conditioning.     

Rapid reaquisition in conditioning of the rabbit's nictitating membrane response.

Reacquisition after extinction often appears faster than original acquisition. However, data from conditioned suppression studies indicate that this effect may arise from spontaneous recovery and reinstatement of unextinguished contextual stimuli related to the unconditioned stimulus (US). In the present experiments using the rabbit nictitating membrane preparation, spontaneous recovery was eradicated before reaquisition training. US contextual stimuli were controlled by retaining the US during extinction through explicit unpairings of the conditioned stimulus (CS) and US. Attempts were also made to drive the associative strength of the CS into the inhibitory region by differential conditioning and conditioned inhibition procedures. In all cases, reacquisition was very rapid in comparison with a rest control. The results are discussed with respect to their implications for CS and US processing models of conditioning.     

Mechanisms underlying retarded emergence of conditioned responding following inhibitory training: evidence for the comparator hypothesis

The comparator hypothesis posits that conditioned responding is determined by a comparison at the time of testing between the associative strengths of the conditioned stimulus (CS) and stimuli proximal to the CS at the time of conditioning. The hypothesis treats all associations as being excitatory and treats conditioned inhibition as the behavioral consequence of a CS that is less excitatory than its comparator stimuli. Conditioned lick suppression by rats was used to differentiate four possible sources of retarded responding to an inhibitory CS. These include habituation to the unconditioned stimulus (US), latent inhibition to the CS, blocking of the CS-US association by the conditioning context, and enhanced excitatory associations to the comparator stimuli. Prior research has demonstrated the first three phenomena. Therefore, we employed parameters expected to highlight the fourth one--the comparator process. In Experiment 1, our negative contingency training was shown to produce a conditioned inhibitor that passed inhibitory summation and retardation tests. In Experiment 2 we found transfer of retardation from an inhibitory CS to a novel stimulus when the location where retardation-test training occurred was excitatory, which is indicative of contextual blocking and/or comparator effects. In Experiment 3, extinction of the conditioning context was found to attenuate retardation regardless of whether extinction occurred before or after the CS-US pairings of the retardation test. This indicates that much of the present retardation was due to the comparator process rather than to contextual blocking. Experiment 4 demonstrated that habituation to the US did not contribute to retardation in the present case. Collectively, these studies suggest that retardation following inhibitory training can be explained without recourse to any of the traditional mechanisms of conditioned inhibition.     

Functional specialization within the striatum along both the dorsal/ventral and anterior/posterior axes during associative learning via reward and punishment

The goal of the present study was to elucidate the role of the human striatum in learning via reward and punishment during an associative learning task. Previous studies have identified the striatum as a critical component in the neural circuitry of reward-related learning. It remains unclear, however, under what task conditions, and to what extent, the striatum is modulated by punishment during an instrumental learning task. Using high-resolution functional magnetic resonance imaging (fMRI) during a reward- and punishment-based probabilistic associative learning task, we observed activity in the ventral putamen for stimuli learned via reward regardless of whether participants were correct or incorrect (i.e., outcome). In contrast, activity in the dorsal caudate was modulated by trials that received feedback--either correct reward or incorrect punishment trials. We also identified an anterior/posterior dissociation reflecting reward and punishment prediction error estimates. Additionally, differences in patterns of activity that correlated with the amount of training were identified along the anterior/posterior axis of the striatum. We suggest that unique subregions of the striatum--separated along both a dorsal/ventral and anterior/posterior axis--differentially participate in the learning of associations through reward and punishment.     

Conditioning across the duration of a backward conditioned stimulus

Five conditioned suppression experiments examined the extent to which an appetitively motivated lever-press response can be punished by different components of a backward conditioned stimulus (CS). Using a 0-s unconditioned stimulus (US)-CS interval, Experiments 1 and 2 showed that the initial 3 s of a normally 30-s backward CS served as a more effective punisher than the CS as a whole. Experiment 3 found no such effect if the US-CS interval were 3 s rather than 0 s. Experiments 4A and 4B found that if the US-CS interval were 0 s, the initial part of the backward CS acquired excitatory properties although the CS as a whole passed a summation test for conditioned inhibition. By contrast, the 3-s US-CS interval supported inhibitory conditioning across the whole duration of the backward CS. Taken together, these findings support a modified version of Wagner's sometimes opponent process model, which suggests that different components of a backward CS become either excitatory or inhibitory depending on the components' temporal proximity to the US.     

Response rates track the history of reinforcement times

When conditioning involves a consistent temporal relationship between the conditioned stimulus (CS) and unconditioned stimulus (US), the expression of conditioned responses within a trial peaks at the usual time of the US relative to the CS. Here we examine the temporal profile of responses during conditioning with variable CS-US intervals. We conditioned stimuli with either uniformly distributed or exponentially distributed random CS-US intervals. In the former case, the frequency of each CS-US interval within a specified range is uniform but the momentary probability of the US (the hazard function) increases as time elapses during the trial; with the latter distribution, short CS-US intervals are more frequent than longer intervals, but the momentary probability of the US is constant across time within the trial. We report that, in a magazine approach paradigm, rats' response rates remained stable as time elapses during the CS when the CS-US intervals were uniformly distributed, whereas their response rates declined when the CS-US intervals were exponentially distributed. In other words, the profile of responding during the CS matched the frequency distribution of the US times, not the momentary probability of the US during the CS. These results are inconsistent with real-time associative models, which predict that associative strength tracks the momentary probability of the US, but may provide support for timing models of conditioning in which conditioned responding is tied to remembered times of reinforcement.     

Facilitation of instrumental behavior by a Pavlovian appetitive conditioned stimulus

Three experiments examined appetitive Pavlovian-instrumental interactions by presenting separately trained conditioned stimuli (CSs) during reinforced instrumental responding in rabbits. Intra-oral reinforcement was used to minimize interference from peripheral responses such as magazine approach. In experiment 1, the rabbits were first trained to perform an instrumental head-raising response for sucrose reward. A conditioned jaw movement response was then established to a 2-sec CS by pairing it with sucrose; a control stimulus was unpaired with sucrose. Instrumental responding maintained by a variable-interval 40-sec schedule was enhanced during 10-sec presentations of the paired, but not the unpaired, CS. Responding on a variable-ratio 15 schedule was unaffected except on trials on which the pre-CS baseline response rate was low; in such cases the paired CS caused a long-lasting acceleration of responding. Noncontingent presentation of the sucrose reinforcer itself briefly suppressed responding but had no long-term effect. In Experiment 2, a CS that had been conditioned at a 10-sec duration produced the same pattern of effects as in the first study, indicating that facilitation resulted from CS presentation rather than from the frustrative effects of non-reinforcement of the CS. In Experiment 3 an inhibitory CS blocked facilitation by the excitatory CS but did not itself affect instrumental responding. These results support the view that Pavlovian processes play a positive role in instrumental performance and suggest that previous findings of suppression by a short-duration CS reflect peripheral interference. The dependence of facilitation on the baseline level of responding is discussed in terms of associative and motivational theories of Pavlovian mediation.     

Variations in explicitly unpaired training are differentially effective in producing conditioned inhibition

To evaluate a contingency interpretation of conditioned inhibition (CI), rats were given “explicity unpaired” training in which the locus and duration of a CS within the inter-US (shock) interval were systematically manipulated for different groups. Summation and retardation tests in Experiment 1 indicated that stronger CI resulted from both a backward and a trace CS than from a midlocus CS of equal or greater duration. Complementing these findings, the same tests in Experiment 2 showed that, by comparison with novel-stimulus controls, CI developed to a trace CS but not to a mid-locus CS, nor to a trace CS that was accompanied by an immediate signal for the US. These findings argue against a contingency interpretation of CI and favor a contiguity interpretation stressing the short-term rehearsal of stimulus events. Such rehearsal of the US allows a backward CS, but not a mid-locus CS with an extended US-CS interval, to be discriminated as a signal for nonreinforcement, and thus to develop as a conditioned inhibitor. Similarly, excitatory conditioning to the memory trace of a CS allows the nominal trace CS to develop as a signal for nonreinforcement, and thus as a conditioned inhibitor, but not when its memory trace is overshadowed by another CS that immediately precedes the US. In short, the development of CI is facilitated when excitation is mediated by the memorial processing of either the US or a discrete CS for the US rather than by contextual cues.     

The role of associative history in models of associative learning: A selective review and a hybrid model

Associative learning theories strive to capture the processes underlying and driving the change in strength of the associations between representations of stimuli that develop as a result of experience of the predictive relationships between those stimuli. Historically, formal models of associative learning have focused on two potential factors underlying associative change, namely processing of the conditioned stimulus (in terms of changes in associability) and processing of the unconditioned stimulus (in terms of changes in error). This review constitutes an analysis of the proper role of these two factors, specifically with regard to the way in which they are influenced by associative history (the prior training undergone by cues). A novel “hybrid” model of associative learning is proposed and is shown to provide a more satisfactory account of the effects of associative history on subsequent learning than any previous single-process theory.     

Opioid receptors mediate direct predictive fear learning: evidence from one-trial blocking

Pavlovian fear learning depends on predictive error, so that fear learning occurs when the actual outcome of a conditioning trial exceeds the expected outcome. Previous research has shown that opioid receptors, including μ-opioid receptors in the ventrolateral quadrant of the midbrain periaqueductal gray (vlPAG), mediate such predictive fear learning. Four experiments reported here used a within-subject one-trial blocking design to study whether opioid receptors mediate a direct or indirect action of predictive error on Pavlovian association formation. In Stage I, rats were trained to fear conditioned stimulus (CS) A by pairing it with shock. In Stage II, CSA and CSB were co-presented once and co-terminated with shock. Two novel stimuli, CSC and CSD, were also co-presented once and co-terminated with shock in Stage II. The results showed one-trial blocking of fear learning (Experiment 1) as well as one-trial unblocking of fear learning when Stage II training employed a higher intensity footshock than was used in Stage I (Experiment 2). Systemic administrations of the opioid receptor antagonist naloxone (Experiment 3) or intra-vlPAG administrations of the selective μ-opioid receptor antagonist CTAP (Experiment 4) prior to Stage II training prevented one-trial blocking. These results show that opioid receptors mediate the direct actions of predictive error on Pavlovian association formation.     

Conditioned inhibition of fear: Evidence for a competing response mechanism

In two experiments, inhibitory conditioning was attempted by presenting a discrete CS in a neutral stimulus environment shortly following the termination of either shock (Experiment 1) or a second discrete CS which had been paired in a forward manner with shock (Experiment 2). Evidence of successful inhibitory conditioning was mixed in Experiment 1, where the properties of the CS were assessed within an escape-from-fear procedure. Postresponse presentations of the CS enhanced performance, whereas the presentation of the CS prior to responding did not have the expected degrading effect on performance. In Experiment 2, the inhibitory properties of the CS were assessed by combining this stimulus with an excitatory CS and presenting the compound to rats engaged in a water-reinforced licking response. Less response suppression was found in reaction to this compound relative to three separate comparison conditions, thus witnessing the success of the inhibitory-conditioning procedure used. The common assumption that inhibitory conditioning results from the nonreinforcement of a CS in a situation where reinforcement is expected, i.e., one which contains previously reinforced cues, is not supported by these data, for no previously reinforced cues were simultaneously presented with the CS during inhibitory training. The data are in agreement with a conditioned antagonistic-response interpretation of inhibitory conditioning.     

Associative representational plasticity in the auditory cortex: a synthesis of two disciplines

Historically, sensory systems have been largely ignored as potential loci of information storage in the neurobiology of learning and memory. They continued to be relegated to the role of "sensory analyzers" despite consistent findings of associatively induced enhancement of responses in primary sensory cortices to behaviorally important signal stimuli, such as conditioned stimuli (CS), during classical conditioning. This disregard may have been promoted by the fact that the brain was interrogated using only one or two stimuli, e.g., a CS(+) sometimes with a CS(-), providing little insight into the specificity of neural plasticity. This review describes a novel approach that synthesizes the basic experimental designs of the experimental psychology of learning with that of sensory neurophysiology. By probing the brain with a large stimulus set before and after learning, this unified method has revealed that associative processes produce highly specific changes in the receptive fields of cells in the primary auditory cortex (A1). This associative representational plasticity (ARP) selectively facilitates responses to tonal CSs at the expense of other frequencies, producing tuning shifts toward and to the CS and expanded representation of CS frequencies in the tonotopic map of A1. ARPs have the major characteristics of associative memory: They are highly specific, discriminative, rapidly acquired, exhibit consolidation over hours and days, and can be retained indefinitely. Evidence to date suggests that ARPs encode the level of acquired behavioral importance of stimuli. The nucleus basalis cholinergic system is sufficient both for the induction of ARPs and the induction of specific auditory memory. Investigation of ARPs has attracted workers with diverse backgrounds, often resulting in behavioral approaches that yield data that are difficult to interpret. The advantages of studying associative representational plasticity are emphasized, as is the need for greater behavioral sophistication.