Neural substrates of olfactory discrimination learning with auditory secondary reinforcement. I. Contributions of the basolateral amygdaloid complex and orbitofrontal cortex

The basolateral amygdaloid complex (BLA) and orbitofrontal cortex (OFC) share extensive reciprocal connections, and interactions between these regions likely contribute to both mnemonic and affective processes. The present study examined the potential differential contributions of the BLA and OFC to performance of an olfactory discrimination task that incorporates auditory conditioned reinforcement and to expression of immediate post-shock freezing behavior. Damage to the BLA had little effect on performance of the conditioned reinforcement task but abolished immediate post-shock freezing behavior. In contrast, damage to OFC resulted in both a mild but significant performance decrement in the conditioned reinforcement task and a significant attenuation of immediate post-shock freezing behavior. These findings suggest that immediate post-shock freezing behavior is likely critically dependent upon interactions between the BLA and OFC. However, although mnemonic processes underlying accurate performance of the conditioned reinforcement task might be supported by OFC in part, such processes are independent of either the BLA or interactions between these two regions.     

Neural Substrates of Olfactory Discrimination Learning with Auditory Secondary Reinforcement. I. Contributions of the Basolateral Amygdaloid Complex and Orbitofrontal Cortex

The basolateral amygdaloid complex (BLA) and orbitofrontal cortex (OFC) share extensive reciprocal connections, and interactions between these regions likely contribute to both mnemonic and affective processes. The present study examined the potential differential contributions of the BLA and OFC to performance of an olfactory discrimination task that incorporates auditory conditioned reinforcement and to expression of immediate post-shock freezing behavior. Damage to the BLA had little effect on performance of the conditioned reinforcement task but abolished immediate post-shock freezing behavior. In contrast, damage to OFC resulted in both a mild but significant performance decrement in the conditioned reinforcement task and a significant attenuation of immediate post-shock freezing behavior. These findings suggest that immediate post-shock freezing behavior is likely critically dependent upon interactions between the BLA and OFC. However, although mnemonic processes underlying accurate performance of the conditioned reinforcement task might be supported by OFC in part, such processes are independent of either the BLA or interactions between these two regions.     

The functional architecture of instrumental conditional reflexes

The author holds the opinion that the so-called operant, or instrumental, conditional reflexes, in spite of differences in specific features, do not differ in principle from classical conditional reflexes, and therefore they should not be opposed to them, or be placed in a special or separate group. In support of this opinion, this paper presents data which show that:

1. 1.
   In the establishment of classical and operant conditional reflexes, two-way conditional connections—direct and reverse—are formed.     
   
   

Differential effects of omission contingencies on various components of Pavlovian appetitive conditioned responding in rats

Five experiments with rat subjects investigated the effects of omission and partial reinforcement contingencies on five individual behaviors evoked by visual and auditory conditioned stimuli paired with a food unconditioned stimulus. The effects of omission depended on the behavior on which that contingency was placed: One behavior was eliminated, one was unaffected, and three were reduced relative to the performance of yoked controls. Partial reinforcement resulted in lower frequencies of three behaviors and higher frequencies of two behaviors, compared with performance under consistent reinforcement. A partial reinforcement extinction effect was noted with one behavior but not with the others. These results are related to the possible role of instrumental conditioning contingencies in generating conditioned behavior in this appetitive conditioning preparation and to the independence of individual components of a complex conditioned response.     

Interaction of dominant focus and conditional reflex as a functional unit of behavior organization

The interaction of two fundamental phenomena—the dominant focus and the conditional reflex—discovered and introduced by A. A. Ukhtomsky and I. P. Pavlov lay at the basis of behavior. According to E. A. Asratyan, the backward conditioned connection is a specialized dominant focus in the functional structure of the consolidated conditional reflex. It makes the behavior goal-directed and active. The dominant focus and conditioned reflex play the same role in the adaptive behavior of the individual as does variability and selection in the process of evolutional adaptation. That is why it is impossible to agree with Popper and Eccles that hypothesis theory has to replace Pavlov’s theory of the conditional reflex. Imprinting and psychonervous activity by images (I. S. Beritashvili) are two special exemplars of conditional reflexes after one coincidence. The so-called “elementary reasoning activity of animals” (according to L. V. Krushinsky) is a kind of the instinctive inherited behavior.     

The influence of experimental neurosis on the conditional reflexes and the content of blood catecholamines and acetylcholine in dogs

Using experimental neurosis as a model, we investigated the mode of adaptive behavior, conditional reflexes and the blood level of neuromediators in four dogs placed in certain versus uncertain conditions in a Pavlovian laboratory. The research consisted of a two year training program with predictable (ordered partial reinforcement) followed by unpredictable (probabilistic reinforcement) situations. As a result, there was a decline in the acetylcholine as compared to a rise of catecholamine levels of the peripheral blood of some of these dogs. There were varied autonomic responses indicating a possible individual response specificity. In one dog, there was a disappearance of motor defense reflexes. The results support the hypothesis that probabilistic reinforcement following ordered partial reinforcement contributed to the dogs’ neurotic disintegration,i.e., uncertainty is a cause of neurotic development.     

The etymology of basic concepts in the experimental analysis of behavior

The origins of many of the basic concepts used in the experimental analysis of behavior can be traced to Pavlov's (1927/1960) discussion of unconditional and conditional reflexes in the dog, but often with substantial changes in meaning (e.g., stimulus, response, and reinforcement). Other terms were added by Skinner (1938/1991) to describe his data on the rate of lever pressing in the rat (e.g., operant conditioning, conditioned reinforcement, negative reinforcement, and response induction and differentiation) and key pecking in the pigeon (shaping). The concept of drive, however, has largely disappeared from the current literature.     

Acquisition of instrumental conditioned reinforcement is resistant to the devaluation of the unconditioned stimulus

The associative mechanisms responsible for the efficacy of Pavlovian stimuli during first- and second-order conditioning have been extensively studied, but little is known about the representations underlying instrumental conditioned reinforcement. The present study investigated the associative structure underlying conditioned reinforcement, by employing an unconditioned stimulus (US) devaluation procedure on a commonly used instrumental task: the acquisition of a new response with conditioned reinforcement. Whilst US-directed behaviour was abolished following devaluation, the conditioned stimulus acting as a conditioned reinforcer supported the acquisition of instrumental responding. In this preparation then, the conditioned reinforcer appears to be impervious to devaluation of its associated US, suggesting that the underlying representation maintaining behaviour is independent of the current value of the US and may reflect the activation of a central appetitive motivational state.     

Limbic-striatal memory systems and drug addiction

Drug addiction can be understood as a pathological subversion of normal brain learning and memory processes strengthened by the motivational impact of drug-associated stimuli, leading to the establishment of compulsive drug-seeking habits. Such habits evolve through a cascade of complex associative processes with Pavlovian and instrumental components that may depend on the integration and coordination of output from several somewhat independent neural systems of learning and memory, each contributing to behavioral performance. Data are reviewed that help to define the influences of conditioned Pavlovian stimuli on goal-directed behavior via sign-tracking, motivational arousal, and conditioned reinforcement. Such influences are mediated via defined corticolimbic-striatal systems converging on the ventral striatum and driving habit-based learning that may depend on the dorsal striatum. These systems include separate and overlapping influences from the amygdala, hippocampus, and cingulate and medial prefrontal cortex on drug-seeking as well as drug-taking behavior, including the propensity to relapse.     

Neuroendocrine and behavioral correlates of constitutional differences

Consistent and stable individual differences were observed in the renal responses of water-loaded dogs during the development of Pavlovian conditional motor defense reflexes by electrocutaneous reinforcement. Some dogs developed persistent and intense conditional antidiuretic responses (with a high urine osmolality) to the entire Pavlovian conditioning room complex, whereas other dogs failed to exhibit conditional antidiuresis. In contradistinction to the conditional motor defense reflexes, the conditional antidiuretic responses showed poor differentiation, thus demonstrating the phenomenon of schizokinesis discovered by Gantt. In addition to conditional water retention, the antidiuretic dogs also exhibited persistent hyperpnea, copious salivation and tachycardia. The physiologic responses of the antidiuretic dogs to the Pavlovian conditioning room resemble those described by Walter B. Cannon for a “fight or flight” response,i.e., physiologic reactions of animals engaged in intense muscular effort. The antidiuresis serves the function of conserving body water so that it may be available for evaporative cooling in order to maintain thermal homeostasis. Since our experimental dogs cannot engage in a consummatory “fight or flight” response, their physiologic reaction to a stressful psychologic environment can be considered maladaptive. Pharmacologic analysis suggested that the antidiuretic responses may involve cholinergic transmitters.     

Studies of instrumental behavior with sexual reinforcement in male rats (Rattus norvegicus): I. Control by brief visual stimuli paired with a receptive female

We describe a novel procedure for measuring instrumental sexual behavior in the male rat by using a second-order schedule of presentation of sexual reinforcement, an estrous female. Experimental assessment and validation of the paradigm have been achieved by examining (a) the importance of the conditioned stimulus in maintaining instrumental responding by measuring the effects of its omission during a test session, (b) the effects and motivational specificity on instrumental behavior of the postejaculatory refractory period (a period of sexual unarousability) and of satiety for food by measuring the impact of each manipulation on animals working for food and for a female, (c) the effects of replacing an estrous female with an anestrous one as the earned reward, and (d) the correlations between conditioned and unconditioned measures of sexual behavior. We conclude that the second-order paradigm provides a means of distinguishing between the effects of neuroendocrine manipulations on incentive motivational and performance variables underlying the expression of sexual behavior.     

A comparison of neuronal reactions during classical and instrumental conditioning under similar conditions

During the elaboration of an instrumental reflex, it is not obligatory to use a conditioned stimulus, which signals the necessity to generate an instrumental reaction in order to receive reinforcement. However, the presence of a conditioned stimulus simplifies analysis of instrumental reaction, which in this case is the response to the conditioned stimulus. On the other hand, it is necessary to distinguish between instrumental and classical conditioning, since in both cases the response to a conditioned stimulus increases. We studied neuronal analogs of classical and instrumental conditioning in the identified neurons responsible for the defensive closure of the pneumostome in the Helix mollusk under the same conditions. During classical conditioning, a mollusk received punishment after a tactile stimulus. During instrumental conditioning, a mollusk received punishment when an identified neuron did not generate an action potential in response to a tactile stimulus. The appearance of a painful stimulus did not depend on the generation or failure of a spike in the related control neuron. Another tactile stimulus, which was never paired with an unconditioned stimulus, was used as a discriminated stimulus. We also compared the behavior of such identified neurons during pseudoconditioning. The experiments were carried out in a semi-intact preparation. We examined how responses to the tactile and painful stimuli changed during different forms of training. It was shown that the dynamics of neuronal responses to a conditioned tactile stimulus were much more complex during instrumental conditioning and consisted of several phases. Throughout a learning session, neural system consecutively acquired information as to which kind of learning was presented, whether a reaction of the neural system must be generated or inhibited and which instrumental reaction is correct. We have demonstrated that response to a painful stimulus during classical conditioning decreases after short-term initial increase. However, during instrumental learning, the neurons controlling instrumental action remained highly sensitive to the unconditioned stimulus. Meanwhile, foreign neurons decreased their responses to the unconditioned stimulus. We may tentatively conclude that classical and instrumental paradigms are fundamentally different at the cellular level.     

Motivational properties of conditioned suppression

Two experiments were performed to investigate the interaction of baseline appetitive drive and incentive on the conditioned suppression of instrumental behavior exhibited in the presence of a preaversive stimulus. In one experiment, suppression of lever pressing of rats working on an intermittent food reinforcement schedule was considerably enhanced following partial satiation with food. In a second experiment, using a similar baseline, suppression was shown to be diminished by an increase in concentration of the baseline incentive. These results are consistent with the hypothesis that the instrumental suppression of the conditioned emotional response (CER) is a consequence of an algebraic interaction of hunger with fear.     

Increased regularity of activity of cortical neurons in learning due to disinhibitory effect of reinforcement

This paper reviews the author’s studies on neurophysiologic mechanisms of conditioned reflex learning. Electroencephalograms, evoked potentials, activity of neocortical and hippocampal neurons and the rabbits’ behavior in the course of elaboration of defensive and inhibitory conditioned reflexes to light flashes have been recorded. Electric shock (ECS) applied to the paw served as reinforcement. The study demonstrated three types of reinforcement effect on the activity of cortical neurons: activating, disinhibitory, and inhibitory. EEG activation due to reinforcement is accompanied by a change in phasic cortical neuronal activity from chaotic or irregular, typical of rest or inhibition, to regular tonic discharges (in neocortex and hippocampus) and group discharges in the stress rhythm, 5–7 Hz in the hippocampus. Following a number of conditioning trials, the effect of reinforcement is simulated by the effect of a conditioned stimulus. With EEG activation and increased regularity in impulses, facilitation of motor reactions is observed.     

Effect of telencephalon ablation on the reinforcing and eliciting properties of species-specific events in Betta splendens

In male Betta splendens, aggressive behavior is drastically attenuated following telencephalon ablation. Because instrumental training and Pavlovian conditioning experiments with intact fish have suggested that associative factors may play an important role in the performance of agonistic behaviors, the effect of ablation on instrumental learning and Pavlovian conditioning was studied. In Experiment 1, ablation had no effect on the learning of the instrumental tunnel-swimming response reinforced by mirror presentation (i.e., viewing a conspecific), although the mirror presentations in yoked-control groups elicited fewer responses in ablates than in normal and sham-operated control fish. Yoked controls further established that instrumental responding was maintained by the reinforcement contingency and was not merely the result of increased motor activity. Experiment 2 studied Pavlovian conditioning of the components of the agonistic display. Unconditioned fin erection, gill erection, and tail beating (i.e., unconditioned responses, URs) to the mirror US all were less frequent in ablates than in normals or shams. Of these, only gill cover erection showed evidence of true conditioning (i.e., conditioned responses; CRs) in which responses to the conditioned stimulus (CS) are due to the pairings of CS and US (unconditioned stimulus). However, ablates suffered no impairment of conditioned gill erections. Ablates performed fewer fin erections to the CS; however, fin erection responses were not due to CS-US pairings but were attributable to pseudoconditioning. These results are related to hypotheses postulating the involvement of learning mechanisms in ablation-produced deficits and normal aggressive behavior.     

Pavlovian influences on goal-directed behavior in mice: the role of cue-reinforcer relations

Two experiments refined procedures to study Pavlovian influences on goal-directed behavior in mice and studied the effects of CS-US relations in Pavlovian-instrumental interactions. Independent groups of mice underwent Pavlovian training to associate either a 10-sec or 2-min auditory stimulus (CS) with reward. We next assessed the ability of the response-contingent CS presentations to reinforce novel instrumental responding (conditioned reinforcement; CRf) or the ability of noncontingent CS presentations to increase ongoing instrumental responding (Pavlovian-instrumental transfer; PIT). Whereas 10-sec training conditions produced strong CRf (and no PIT), 2-min training conditions produced robust PIT (but no CRf).     

The problem of switching in conditional behavior

This article reviews studies of various authors on the phenomenon of “switching,” which is observed in both classical and instrumental conditioning and consists in elicitation of different responses to the same conditional stimulus (CS) when it is applied in an environment different than the original one. The different responses include a decrease or an absence of the previously trained conditional response (CR), elicitation of an appetitive response instead a defensive one, or vice versa, as well as elicitation of two different instrumental CRs in the same trial. The studies suggest that, due to the repeated occurrence of CS in the same environment (E), also called “situation” or “context,” associations are formed between CS and E. Consequently, the CR is elicited to a compound CS+E rather than to CS alone. When the CS is applied alone in a different E than the original one, the previously formed associations are inactive and the CR cannot be elicited; this leads to switching. Studies also suggest that E plays a dominant role in conditioning compared with that of CS alone, which often appears to be only a trigger for eliciting the response. However, CS tested in a different E may still produce some components of the previously acquired CR, such as a general fear behavior to an originally defensive CS or an approach behavior to an originally alimentary CS. The environmental stimuli can be considered the “determining” stimuli that determine the kind of reaction to be elicited, or “tonic” stimuli that increase the tonus in the brain but do not elicit the CR. The “determining” or “tonic” stimuli do not seem to be a special class of stimuli. Instead, they are stimuli that initially can produce the CR (e.g., intertrial CRs), but by being not reinforced they become partly inhibited; nevertheless, due to associations with the reinforcement, they still can produce some excitement related to it, thus facilitating the CR.     

An architect of the golden years

William H. Morse has played a major role in the experimental analysis of behavior. His view of operant behavior as the outcome of differential reinforcement provides an invaluable lesson in scientific research and theory. He studied schedules of reinforcement to generate an in-depth analysis of the complex interactions existing when contingencies exert their control over behavior. He has been instrumental in showing how behavior is determined by the dynamic interaction of factors brought into play by the imposition of any schedule, and he has a remarkably intuitive understanding of the nature of these determining variables. Some of these causal events are imposed directly by the schedule, but others arise in a more indirect manner through necessary constraints. In Morse's view, schedules can be more fundamental in determining behavior than are the scheduled events themselves. Behavior is the shaped product of an organism's history in combination with present environmental conditions. His impact deserves to be more than historical: A study of his work continues to reward the reader with exciting insights into the nature of behavioral control.     

The bioelectrical activity of cerebellar structures in the freely-moving rat

Electrodes were implanted in the brains of 27 freelymoving rats and the bioelectrical activity of cerebellar cortical structures (lobus simplex, tuber verrais, lobus ansiformis, crus II) and the dentate nucleus was measured simultaneously with the activity of the cerebral cortex and dorsal hippocampus and respiratory rate and motor activity. Different behavioral states were produced by habituation procedures and by elaborating conditional avoidance reflexes to light-flash or click series. In addition, startle reflexes to acoustic stimuli were used to evaluate behavioral state. The following conclusions could be drawn: 1) in the awake rat the various cerebellar structures have clearly distinguishable bioelectrical activity patterns; 2) changes in these patterns depend on the actual behavioral state of the animal; 3) this dependence upon behavior is especially clear in crus II, one of the projection zones of the tactile and proprioceptive afferent nerves in the cerebellar cortex. The changes in the electrocerebellogram of the unrestrained rat may be used as an indicator of the behavioral state of the animal.     

A selectionist approach to reinforcement.

We describe a principle of reinforcement that draws upon experimental analyses of both behavior and the neurosciences. Some of the implications of this principle for the interpretation of behavior are explored using computer simulations of adaptive neural networks. The simulations indicate that a single reinforcement principle, implemented in a biologically plausible neural network, is competent to produce as its cumulative product networks that can mediate a substantial number of the phenomena generated by respondent and operant contingencies. These include acquisition, extinction, reacquisition, conditioned reinforcement, and stimulus-control phenomena such as blocking and stimulus discrimination. The characteristics of the environment-behavior relations selected by the action of reinforcement on the connectivity of the network are consistent with behavior-analytic formulations: Operants are not elicited but, instead, the network permits them to be guided by the environment. Moreover, the guidance of behavior is context dependent, with the pathways activated by a stimulus determined in part by what other stimuli are acting on the network at that moment. In keeping with a selectionist approach to complexity, the cumulative effects of relatively simple reinforcement processes give promise of simulating the complex behavior of living organisms when acting upon adaptive neural networks.