A comparison of neuronal reactions during classical and instrumental conditioning under similar conditions

During the elaboration of an instrumental reflex, it is not obligatory to use a conditioned stimulus, which signals the necessity to generate an instrumental reaction in order to receive reinforcement. However, the presence of a conditioned stimulus simplifies analysis of instrumental reaction, which in this case is the response to the conditioned stimulus. On the other hand, it is necessary to distinguish between instrumental and classical conditioning, since in both cases the response to a conditioned stimulus increases. We studied neuronal analogs of classical and instrumental conditioning in the identified neurons responsible for the defensive closure of the pneumostome in the Helix mollusk under the same conditions. During classical conditioning, a mollusk received punishment after a tactile stimulus. During instrumental conditioning, a mollusk received punishment when an identified neuron did not generate an action potential in response to a tactile stimulus. The appearance of a painful stimulus did not depend on the generation or failure of a spike in the related control neuron. Another tactile stimulus, which was never paired with an unconditioned stimulus, was used as a discriminated stimulus. We also compared the behavior of such identified neurons during pseudoconditioning. The experiments were carried out in a semi-intact preparation. We examined how responses to the tactile and painful stimuli changed during different forms of training. It was shown that the dynamics of neuronal responses to a conditioned tactile stimulus were much more complex during instrumental conditioning and consisted of several phases. Throughout a learning session, neural system consecutively acquired information as to which kind of learning was presented, whether a reaction of the neural system must be generated or inhibited and which instrumental reaction is correct. We have demonstrated that response to a painful stimulus during classical conditioning decreases after short-term initial increase. However, during instrumental learning, the neurons controlling instrumental action remained highly sensitive to the unconditioned stimulus. Meanwhile, foreign neurons decreased their responses to the unconditioned stimulus. We may tentatively conclude that classical and instrumental paradigms are fundamentally different at the cellular level.     

Transfer of control between causal predictive judgments and instrumental responding

Four experiments were conducted to explore outcome-specific transfer from causal predictive judgments to instrumental responding. A video game was designed in which participants had to defend Andalusia from navy and air force attacks. First, they learned the relationship between two instrumental responses (two key on a standard keyboard) and two different outcomes (destruction of the ships or destruction of the planes). Then they learned to predict which of two different stimuli predicted which outcome. Finally, they had the opportunity of making either of the two instrumental responses in the presence of either stimulus. Transfer was shown as a preference for the response that shared an outcome with the current stimulus. The presentation of the stimulus during the test produced a decrease in the overall rate of response. Responding to a neutral stimulus in Experiments 2 and 3 suggested that this overall decrease in responding was due to a combination of the time needed to process the meaning of the stimulus and the activation of the representation of the outcome in the presence of the stimulus during the test. Transfer between predictive judgments and instrumental responding mirrors the outcome-specific Pavlovian instrumental transfer observed in conditioning studies with rats.     

Discrimination between reinforced action patterns in the rat

Laboratory rats were rewarded for face-washing, rearing, or scratching by being given the opportunity to press retractable levers for food reward. Yoked control animals received the same number of lever presentations and food rewards, but did not have to face-wash, rear, or scratch to obtain the levers. The experimental animals showed increases in number of bouts of reinforced target behavior above control levels, and the total amount of time spent face-washing increased when a 1.5-sec criterion for reinforced bout length was introduced. The activities in this experiment were made to serve a discriminative as well as an instrumental function, since the cue to tell the rat which lever to press for reward when the levers were presented was the activity that the rat had engaged in to obtain lever presentation. In two separate experiments high levels of discrimination between behaviors were obtained. Discrimination was worse following scratching than after other actions, and scratching also showed relatively poor instrumental conditioning. The relationship between Pavlovian and instrumental conditioning processes in this situation is discussed.     

Observation and discrimination learning in the rat: Effects of stimulus substitution

Effects of prior observation of modeling performances upon learning simultaneous visual discriminations were studied in nine groups of rats. Stimulus patterns which had been positive (“Cued”) or negative (“Uncued”) for the model animal and novel stimuli were presented. Observers learned faster if discriminations required approach to, rather than avoidance of, the previously positive stimulus. Approach to the negative stimulus was acquired more rapidly only if the second stimulus was novel. Approaches to a substituted new stimulus combined with either of the previously viewed stimulus patterns required nearly twice the training needed for a similar response attachment when both stimuli were novel. Attentional changes appear to underlie the stimulus control exerted over the observer rat in instrumental learning situations.     

Length of experimental session as a parameter of response rate under a non-discriminated

Following the stabilization of response rate under an avoidance schedule which was defined by two temporal parameters, the shock-shock interval and the interval by which each response postponed the onset of shock, the length of the experimental session was changed. It was found that after the subjects had been exposed to a longer session of avoidance schedule, their rates of response were considerably increased without a corresponding reduction in the number of shocks received.

In recent years considerable use has been made of an avoidance training technique in which the performance of the response functions to postpone the onset of an aversive stimulus, usually shock, by a fixed period. In the absence of the required response the aversive stimulus is programmed to occur at regular intervals. Experiments by Sidman (1953) have shown that the critical independent variable controlling the rate of avoidance response, is the shock postponement interval (R*S). All other things being equal, the rat in the lever pressing situation will respond at a rate which is inversely related to the R*S interval, low intervals generating high response rates and high intervals generating low response rates. However, under very low values of R*S, the response rate may break down altogether. The animal then receives shock at the rate determined by the shock-shock interval parameter.

As a result of an apparatus failure, Sidman, Herrnstein and Conrad (1957) discovered that the response rate can also be increased by occasionally shocking the animal in spite of its avoidance responses. An apparatus failure has also been responsible for the isolation of yet another parameter of response rate in the shock-postponement avoidance situation and is reported here. Briefly, it was found that a change in the duration of an experimental session influences the response rate on subsequent sessions.     

The overshadowing and blocking of punishment

Two experiments were undertaken to test the assumption that the outcome of an instrumental contingency depends on the acquisition of signalling properties by the response in the same sense that the acquisition of signalling properties by a stimulus is viewed as determining the outcome of a Pavlovian contingency. The presence coincident with a punished response of a stimulus either previously or preferentially correlated with shock was found to reduce the extent to which punishment suppressed the rate of responding. It was argued that these results, interpreted as indicating the blocking and overshadowing of the response-shock association, were consistent with the assumption tested.     

Avoidance extinction in goldfish

Independent groups of goldfish trained to avoid shock in a shuttlebox situation were presented with several extinction procedures in which the relationships between the conditioned stimulus and shock were altered and/or response contingencies removed. Random shock presentations, equivalent to the number of shocks received during avoidance acquisition, resulted in response decrements similar to those obtained when the conditioned stimulus was presented alone. Pairing the conditioned stimulus with shock on every trial, however, served to maintain response levels. When response-contingent punishment was superimposed upon these Pavlovian pairings, performance was facilitated slightly although punishment alone resulted in somewhat faster response reduction than that produced by exposure only to the conditioned stimulus. Extinction of avoidance responding produced by exposure to the conditioned stimulus alone was dependent on the total duration of exposure and independent of both number of stimulus onsets and response prevention. These experiments demonstrated that, in general, the procedures used to reduce avoidance responding in rats were equally effective for goldfish, with one exception: the introduction of a Pavlovian contingency following avoidance acquisition, making the previously avoidable shock unavoidable, maintained response probabilities near previously established levels.     

Response inhibition in Extinction

In three experiments, rats were given a nonreinforced choice between two instrumental responses, in the presence of stimuli that had separately signalled the various outcomes those responses had earned. Whether those stimuli had been trained in a Pavlovian or instrumental procedure, they selectively enhanced the transfer response with which they shared an outcome. However, a subsequent test showed that this greater responding to a stimulus during extinction produced a reduced ability of that stimulus to augment its transfer response. These results are consistent with the development of an inhibitory S-R association during extinction.     

Blocking in successive differential conditioning: Prior acquisition of control by internal cues blocks the acquisition of control by brightness

In each of four experiments the acquisition of discriminative stimulus control by brightness cues (black vs white runway) in a successive go/no-go instrumental discrimination was blocked in groups given prior discrimination training with internal (reward-produced and intertrial interval-related) cues as relevant discriminanda and brightness cues irrelevant. The blocking effects obtained here in instrumental conditioning were substantial and in most cases complete. Blocking occurred whether brightness in Phase I varied across trials but was uncorrelated with reinforcement (varied-irrelevant, V-I, condition) or whether all Phase 1 trials occurred to a single value on the brightness dimension (constant-irrelevant, C-I, condition) which then served as a redundant S+ cue in Phase 2 while the previously unexperienced brightness cue was added to the S- stimulus compound, or vice versa. Blocking in the V-I condition was shown not to be due simply to nondifferential reinforcement of brightness in Phase 1, but to depend on the prior acquisition of discriminative stimulus control by internal cues. As in Pavlovian conditioning, blocking here was an increasing function of amount of prior conditioning to the blocking stimulus. The results encourage the prospect that the procedures used here can be developed into a viable instrumental conditioning companion to the Pavlovian procedures now used almost exclusively to study blocking.     

Motivational properties of conditioned suppression

Two experiments were performed to investigate the interaction of baseline appetitive drive and incentive on the conditioned suppression of instrumental behavior exhibited in the presence of a preaversive stimulus. In one experiment, suppression of lever pressing of rats working on an intermittent food reinforcement schedule was considerably enhanced following partial satiation with food. In a second experiment, using a similar baseline, suppression was shown to be diminished by an increase in concentration of the baseline incentive. These results are consistent with the hypothesis that the instrumental suppression of the conditioned emotional response (CER) is a consequence of an algebraic interaction of hunger with fear.     

Facilitation of instrumental behavior by a Pavlovian appetitive conditioned stimulus

Three experiments examined appetitive Pavlovian-instrumental interactions by presenting separately trained conditioned stimuli (CSs) during reinforced instrumental responding in rabbits. Intra-oral reinforcement was used to minimize interference from peripheral responses such as magazine approach. In experiment 1, the rabbits were first trained to perform an instrumental head-raising response for sucrose reward. A conditioned jaw movement response was then established to a 2-sec CS by pairing it with sucrose; a control stimulus was unpaired with sucrose. Instrumental responding maintained by a variable-interval 40-sec schedule was enhanced during 10-sec presentations of the paired, but not the unpaired, CS. Responding on a variable-ratio 15 schedule was unaffected except on trials on which the pre-CS baseline response rate was low; in such cases the paired CS caused a long-lasting acceleration of responding. Noncontingent presentation of the sucrose reinforcer itself briefly suppressed responding but had no long-term effect. In Experiment 2, a CS that had been conditioned at a 10-sec duration produced the same pattern of effects as in the first study, indicating that facilitation resulted from CS presentation rather than from the frustrative effects of non-reinforcement of the CS. In Experiment 3 an inhibitory CS blocked facilitation by the excitatory CS but did not itself affect instrumental responding. These results support the view that Pavlovian processes play a positive role in instrumental performance and suggest that previous findings of suppression by a short-duration CS reflect peripheral interference. The dependence of facilitation on the baseline level of responding is discussed in terms of associative and motivational theories of Pavlovian mediation.     

Transfer of orthogonal stimulus–response mappings to an orthogonal Simon task

When up–down stimulus locations are mapped to left–right keypresses, an overall advantage for the up–right/down–left mapping is often obtained that varies as a function of response eccentricity. This orthogonal stimulus–response compatibility (SRC) effect also occurs when stimulus location is irrelevant, a phenomenon called the orthogonal Simon effect, and has been attributed to correspondence of stimulus and response code polarities. The Simon effect for horizontal stimulus–response (S–R) arrangements has been shown to be affected by short-term S–R associations established through the mapping used for a prior SRC task in which stimulus location was relevant. We examined whether such associations also transfer between orthogonal SRC and Simon tasks and whether correspondence of code polarities continues to contribute to performance in the Simon task. In Experiment 1, the orthogonal Simon effect was larger after practising with an up–right/down–left mapping of visual stimuli to responses than with the alternative mapping, for which the orthogonal Simon effect tended to reverse. Experiment 2 showed similar results when practice was with high (up) and low (down) pitch tones, though the influence of practice mapping was not as large as that in Experiment 1, implying that the short-term S–R associations acquired in practice are at least in part not modality specific. In Experiment 3, response eccentricity and practice mapping were shown to have separate influences on the orthogonal Simon effect, as expected if both code polarity and acquired S–R associations contribute to performance.     

Escape from fear: a detailed behavioral analysis of two atypical responses reinforced by CS termination

Escape from fear (EFF) is a controversial paradigm according to which animals learn to actively escape a fear-eliciting conditioned stimulus (CS) if the escape response (R-sub(e)) is paired with CS termination. Some theories posit that EFF learning is responsible for instrumental avoidance conditioning. However, EFF learning has typically been weaker than avoidance learning and difficult to reproduce. The authors examined EFF learning and memory with 2 atypical R-sub(e)s: rearing and nose-poking. The data suggest that rearing, but not nose-poking, can be learned as an instrumental EFF response. Further, EFF memory was response specific, aversively motivated, and controlled by the CS. Successful EFF learning also resulted in better long-term elimination of a passive fear reaction (freezing). Factors important for EFF experiments and theoretical considerations are discussed.     

The roles of stimulus control and reinforcement frequency in modulating the behavioral effects of d-amphetamine in the rat.

The behavioral effects of d-amphetamine have been shown to be modulated by stimulus control, with less impairment of performance occurring when control is great. When the fixed-consecutive-number schedule is used (on which at least a specified consecutive number of responses must be made on one operandum before a single response on another will produce a reinforcer), response rate tends to be invariant but reinforcement frequency is not. This study asks whether the differences in reinforcement frequency that usually accompany changes in stimulus control could themselves be responsible for the performance differences. Two versions of the fixed-consecutive-number schedule of reinforcement were combined into a multiple schedule within which stimulus control was varied but differences in reinforcement frequency were minimized by omitting some reinforcer deliveries during the component that usually had the higher reinforcement frequency. In one component, a compound discriminative stimulus was added with the eighth consecutive response on the first lever; a single response on the second lever was then reinforced. In the other component, no such stimulus was presented. With no added stimulus, large decreases occurred in the number of runs satisfying the minimum requirement for reinforcement at doses of drug that produced only minimal changes when an added stimulus controlled behavior. Thus, increased stimulus control diminishes the behavioral changes produced by d-amphetamine even when the possible contribution by baseline reinforcement rate is minimized.     

Associations in human instrumental conditioning

Four experiments were conducted to study the contents of human instrumental conditioning. Experiment 1 found positive transfer between a discriminative stimulus (S^D) and an instrumental response (R) that shared the outcome (O) with the response that was originally trained with the S^D, showing the formation of an S^D → O association. Experiment 2 found that post-acquisition devaluation of an outcome selectively reduced the response trained with that outcome, showing the formation of a R → O association. In Experiment 3, changing the outcome did not prevent participants from giving the response learned with each S^D, even though none of the responses was appropriate for the new outcome, showing evidence of the formation of S^D → R associations in instrumental learning. The three binary associations were shown within the same basic experimental situation. Finally, Experiment 4 found evidence of the formation of the higher order association S^D(R → O) in human instrumental conditioning.     

The peripheral unconditioned stimulus pathway in a model learning system involving defensively conditioned heart rate change in the pigeon.

The peripheral pathway transmitting the unconditioned stimulus information is described in a vertebrate model system for studying learning-visually conditioned heart rate change in the pigeon. In this system footshock serves as the unconditioned stimulus, and it is shown that the effective stimulation field of this stimulus, defined with respect to the unconditioned cardioacceleratory response, includes cutaneous dermatomes 21-25 (L1-Sl). The stimulus information is then transmitted by Group III and IV fibers in portions of the femoral and sciatic nerves which enter the spinal cord over segments 21-25. Further, it is shown that interrupting this pathway by section of dorsal roots 21-25 precludes establishing conditioned heart rate change with footshock as the unconditioned stimulus.     

Analysis of treatments to alleviate forgetting in rats

Two experiments with rats investigated the effectiveness of prior-cuing treatments for alleviating forgetting of aversive conditioning. The aim was to see which retrieval cues would be most effective within different contexts. Experiment 1 examined the contexts of classical fear conditioning and instrumental avoidance training. The results indicated that the response components were sufficient to reinstate avoidance training, whereas the unconditioned stimulus (US) was most effective for classical fear conditioning. In Experiment 1, the reinforcer per se was ineffective in reinstating instrumental avoidance training. Experiment 2 manipulated the training context and found that the US could be made an effective prior-cuing treatment for instrumental training if classical conditioning components were more prevalent during training. These results are interpreted to mean that a "critical context" must be reinstated by the cuing treatment if this treatment is to promote retrieval of the memory.     

Resolution of outcome-induced response conflict by humans after extended training

Studies of incongruent discrimination learning, where the outcome event of one response acts as the discriminative stimulus for the opposite response, suggest that humans rely on habitual stimulus–response (S–R) associations when outcome–response (O–R) associations would cause response conflict. Here, two experiments were conducted to investigate the robustness of this habitual strategy. In Experiment 1, we found that extensive instrumental discrimination training supported learning about the incongruent R → O contingencies, as assessed by an outcome devaluation test. Differential representations of the stimulus and the (associatively retrieved) outcome may have allowed for goal-directed incongruent performance. Experiment 2 failed to provide evidence for this possibility; direct presentation as well as associative retrieval of the incongruent events (by Pavlovian stimuli) activated the response that was associated with each event in its role of stimulus as opposed to outcome. We did find that participants successfully acquired explicit knowledge of the incongruent contingencies, which raises the possibility that propositional encoding allowed them to overcome the response conflict caused by O–R associations. Alternative associative and propositional accounts of successful goal-directed incongruent performance with extensive training will be discussed.     

An analysis of contrast effects in multiple schedules

Some phenomena of behavioral contrast in multiple schedules are reviewed, and three accounts of contrast are considered. Rate changes within a constant schedule component (transient contrasts) are distinguished from rate changes across successive schedule cycles (sustained contrasts). Pigeons were exposed to a three-component multiple schedule, in which a stimulus correlated with a constant variable interval schedule of reinforcement was preceded by a stimulus correlated with more frequent variable interval reinforcement, or by an extinction stimulus. If the preceding stimulus was correlated with more frequent reinforcement, the response rate in the constant component was low and increased with time. If the preceding stimulus was correlated with extinction, the rate in the constant component was high and decreased with time. Similar transient contrasts were observed in a two-component multiple schedule with different variable interval schedules in the two components. The transient contrast effects in the three-component schedule were shown to depend on differential reinforcement frequency rather than differential response rate in the preceding component. Such transient contrasts were not sufficient to account for sustained contrast effects observed in these experiments. The relation of these findings to the concepts of excitation and inhibition is discussed.