Identification of a reinforcement pathway necessary for operant conditioning of head waving in Aplysia californica

The marine mollusc Aplysia californica exhibits a wide range of nonassociative and associative forms of learning. Recently, we found that the learning repertoire of Aplysia includes operant conditioning (Cook & Carew, 1986, 1989b). The behavior we examined is a naturally occurring, side-to-side head-waving response used by Aplysia in seeking food, obtaining a foothold, and egg laying. Aplysia can be operantly conditioned to reduce head-waving to one side of their body if such a response results in exposure to bright uniform-field illumination, which the animals find aversive. An essential step toward achieving a mechanistic understanding of operant conditioning is to identify and characterize the reinforcement pathway used during the learning. Toward this end, we wished to determine which of the peripheral visual pathways in Aplysia are critical for performance of the operant task. Previous experiments indicated that photic input from the optic and rhinophore nerves functionally inhibited motor neurons that participate in the operant response (head-waving), while photic input from the oral veil nerves excited these same motor neurons (Cook & Carew, 1989c). These findings suggested the hypothesis that one or both of these pathways could play an important role in mediating reinforcement during training. To explore this possibility we operantly trained animals that had received chronic bilateral transections of either the optic and rhinophore nerves or the oral veil nerves C1-C3 (in conjunction with transection of the optic and rhinophore nerves). We found that operant conditioning was not disrupted by ablation of input from the eyes and rhinophores. By contrast, ablation of input from the oral veil (together with that from the eyes and rhinophores) abolished operant conditioning. Thus, the oral veil nerves play a critical modulatory role in operant conditioning of head-waving. This observation further suggested that photic input from the oral veil is conveyed to the CNS via the oral veil nerves. In a final experiment we confirmed that stimulation of the oral veil with light evokes increased afferent activity in the oral veil nerves C1-C2. These results support the idea that the oral veil nerves contain processes that are critical components of the reinforcement pathway for operant conditioning of head-waving.     

An Analysis of Feedback from a Behavior Analytic Perspective

The present paper presents a systematic analysis from a behavior analytic perspective of procedures termed feedback. Although feedback procedures are widely reported in the discipline of psychology, including in the field of behavior analysis, feedback is neither consistently defined nor analyzed. Feedback is frequently treated as a principle of behavior; however, its effects are rarely analyzed in terms of well-established principles of learning and behavior analysis. On the assumption that effectiveness of feedback procedures would be enhanced when their use is informed by these principles, we sought to provide a conceptually systematic account of feedback effects in terms of operant conditioning principles. In the first comprehensive review of this type, we compare feedback procedures with those of well-defined operant procedures. We also compare the functional relations that have been observed between parameters of consequence delivery and behavior under both feedback and operant procedures. The similarities observed in the preceding analyses suggest that processes revealed in operant conditioning procedures are sufficient to explain the phenomena observed in studies on feedback.     

The operant and the classical in conditioned orientation of Drosophila melanogaster at the flight simulator

Ever since learning and memory have been studied experimentally, the relationship between operant and classical conditioning has been controversial. Operant conditioning is any form of conditioning that essentially depends on the animal's behavior. It relies on operant behavior. A motor output is called operant if it controls a sensory variable. The Drosophila flight simulator, in which the relevant behavior is a single motor variable (yaw torque), fully separates the operant and classical components of a complex conditioning task. In this paradigm a tethered fly learns, operantly or classically, to prefer and avoid certain flight orientations in relation to the surrounding panorama. Yaw torque is recorded and, in the operant mode, controls the panorama. Using a yoked control, we show that classical pattern learning necessitates more extensive training than operant pattern learning. We compare in detail the microstructure of yaw torque after classical and operant training but find no evidence for acquired behavioral traits after operant conditioning that might explain this difference. We therefore conclude that the operant behavior has a facilitating effect on the classical training. In addition, we show that an operantly learned stimulus is successfully transferred from the behavior of the training to a different behavior. This result unequivocally demonstrates that during operant conditioning classical associations can be formed.     

Elicited versus emitted behavior: Time to abandon the distinction

The concept of emitted behavior was formulated as a part of the original argument for the validity of a new kind of learning called operant conditioning. The rationale for operant conditioning contrasted it with Pavlovian or classical conditioning, which was (and remains) fundamentally based on responses to conditioned and unconditioned stimuli. Classical conditioned responses were said to be elicited. In contrast, operant behavior was viewed as emitted and controlled primarily by response consequences rather than antecedents. I argue that the distinction between emitted and elicited behavior is no longer warranted for three major reasons. First, the distinction was based on a view of Pavlovian conditioning that is no longer viable. Second, the distinction is incompatible with both empirical data and contemporary conceptualizations of operant behavior. Third, the only way to overcome these problems is to define emitted and elicited in terms of the type of conditioning (operant and classical) that produces these behaviors, but that approach makes the definitions circular and does not avoid implications of the terms that are misleading and counterproductive in light of contemporary research and thinking.     

The infant vocal-conditioning literature: a theoretical and methodological critique

Most studies of operant conditioning of infant vocalization do not fully use the experimental-analysis-of-behavior methodology that behavioral researchers have developed to study operant phenomena. This could be a contributing factor in their failure to correctly operationalize the definition of reinforcement, severely limiting the amount of information they can provide about operant learning in infants. Furthermore, single-subject-design studies may be added to supplement or replace group experimental designs in the study of infant vocal conditioning if we are to recognize the different learning processes that affect infant learning. Finally, single-subject experimental designs may be crucial to the development of an effective technology of early language intervention.     

Determinants of human operant heart rate conditioning: a systematic investigation of several methodological issues.

Three methodological issues of concern within the literature on human operant heart rate conditioning were assessed utilizing a number of techniques for data reduction. The type of statistical approach largely determined the conclusions to be drawn about 2 issues: (a) differences and similarities between heart rate acceleration and deceleration learning and (b) changes in conditioning over a number of sessions. Three techniques yielded data that confounded between- and within-session shifts in tonic heart rate. A fourth method of data reduction (contrasts between pretrial and trial responding) involved no such difficulty. This method of analysis most accurately represented changes in operant heart rate that occurred within and between conditioning sessions.     

Is behaviorism false because there is no such thing as conditioning? Popper and skinner on learning

Sir Karl Popper has claimed that behaviorism is misguided because it holds that conditioning occurs through repetition. According to Popper, there is no such thing as learning through repetition. To the limited extent that philosophers of science have concerned themselves with behaviorism, this attack is one of the most direct and unique in that the battleground is not over the value of mentalism/cognitivism but a bold claim that conditioning—the heart and soul of behaviorism—is fictitious. This paper examines the soundness of Popper's argument against behaviorism by examining whether operant and classical conditioning rely on learning through repetition and suffer from other problems Popper attributes to the notion of conditioning. Although Popper correctly attributes certain properties to classical conditioning, he fails to undermine the empirical evidence that such conditioning occurs. Second, we claim that although B. F. Skinner is never entirely clear, operant conditioning does not rely on repetition nor does it suffer from the other problems Popper attributes to conditioning. Thus, Popper's argument also fails because of his assumption that all conditioning is classical conditioning and therefore his misattributing properties of classical conditioning to operant conditioning. We conjecture that the earlier polemics of John Watson and Watson's sole reliance on classical conditioning probably contributed to Popper's confusion on this point.     

Evaluation of infant vocal conditioning.

The use of the operant conditioning paradigm, as it has been applied to infant social, vocal behavior, reflects a failure to take into account the social nature of human infants over and above the rigid theoretical rationale of the paradigm. It is argued that: (1) the baseline procedure used in operant conditioning studies is methodologically and conceptually invalid; (2) the reinforcing stimulus used in social conditioning studies elicits the very response that it is assigned to reinforce; and (3) the effect of the response-reinforcer relationship is not the reinforcement (strengthening) of vocal response rate.     

Feeding behavior of Aplysia: a model system for comparing cellular mechanisms of classical and operant conditioning

Feeding behavior of Aplysia provides an excellent model system for analyzing and comparing mechanisms underlying appetitive classical conditioning and reward operant conditioning. Behavioral protocols have been developed for both forms of associative learning, both of which increase the occurrence of biting following training. Because the neural circuitry that mediates the behavior is well characterized and amenable to detailed cellular analyses, substantial progress has been made toward a comparative analysis of the cellular mechanisms underlying these two forms of associative learning. Both forms of associative learning use the same reinforcement pathway (the esophageal nerve, En) and the same reinforcement transmitter (dopamine, DA). In addition, at least one cellular locus of plasticity (cell B51) is modified by both forms of associative learning. However, the two forms of associative learning have opposite effects on B51. Classical conditioning decreases the excitability of B51, whereas operant conditioning increases the excitability of B51. Thus, the approach of using two forms of associative learning to modify a single behavior, which is mediated by an analytically tractable neural circuit, is revealing similarities and differences in the mechanisms that underlie classical and operant conditioning.     

Defining reinforcess — A problem in communication for consultation in behavior modification

This paper reviews the respondent (Hull-Spence) and operant (Skinnerian) conditioning definitions of reinforcers and reinforcement and demonstrates the need to keep the systems separate when consulting about behavior modification. The two systems are shown to lead to different modification procedures.One important distinction between the systems is whether a reinforcer is simply associated with a response (respondent) or whether it must follow the response (operant). A second important distinction is the definition of negative reinforcement. In respondent conditioning, negative reinforcement entails presenting an aversive stimulus in association with the response and results in a decrease in response rate. In operant conditioning, negative reinforcement entails the removal of an aversive stimulus following a correct response, which results in an increase in response rate.     

Voluntary Blood Pressure Control: Operant Conditioning by a Continuous Blood Pressure Feedback Technique

Abstract

The present study evaluated a single session operant conditioning of blood pressure in borderline hypertensives. A double blind randomized two period (each lasting for 20 minutes) cross over study consisting of a treatment (operant conditioning of mean arterial blood pressure) and a control condition (heart rate) was performed. Sequence of treatments was randomly assigned. Fourteen borderline hypertensive subjects participated and were trained to decrease their continuously measured blood pressure (Finapres) and heart rate. Subjects receiving blood pressure feedback during the first session showed a significant (4.3 mmHg) decrease in diastolic blood pressure. In contrast, subjects receiving blood pressure training during the second session showed a non-significant increase in diastolic blood pressure. Treatment consisting of more sessions is a promising adjunct to antihypertensive regimens.     

Operant techniques used in stuttering therapy: A review

This article reviews research concerning the use of operant conditioning in stuttering therapy, and discusses the clinical implications of this literature. In order to be considered for this review, a report had to treat the findings of research specifically designed to use operant conditioning in the manipulation of speech disfluency. This body of experimental literature clearly indicates that operant management techniques can effectively reduce stuttering with punishment of disfluencies producing more notable results than reinforcement of fluent responses. Operant conditioning is an effective means of modifying stuttering behavior and should be more visible in terms of procedures used by speech therapists.     

Extending in vitro conditioning in Aplysia to analyze operant and classical processes in the same preparation

Operant and classical conditioning are major processes shaping behavioral responses in all animals. Although the understanding of the mechanisms of classical conditioning has expanded significantly, the understanding of the mechanisms of operant conditioning is more limited. Recent developments in Aplysia are helping to narrow the gap in the level of understanding between operant and classical conditioning, and have raised the possibility of studying the neuronal processes underlying the interaction of operant and classical components in a relatively complex learning task. In the present study, we describe a first step toward realizing this goal, by developing a single in vitro preparation in which both operant and classical conditioning can be studied concurrently. The new paradigm reproduced previously published results, even under more conservative and homogenous selection criteria and tonic stimulation regime. Moreover, the observed learning was resistant to delay, shortening, and signaling of reinforcement.     

The Wandering IQ: A Continuation

The hypothesis addressed in this study was that for every experimental condition, learning with awareness would occur. An experiment was conducted with use of a collating task that incorporated the awareness question into the design investigating the change in performance quality and quantity. Statistical evaluation of Dulany's awareness questionnaire suggested general unawareness of attempted verbal operant conditioning. There is some evidence suggesting that an actual change in performance quality and quantity is accompanied by awareness with use of operant conditioning procedures.     

Measuring the quality of residential care

Administrative personnel require objective, reliable and valid information in order to monitor the effects of allocating resources to different components of the service that they are providing. The initial problem is to define and measure the dependent variable—the ‘quality of care’ —against which the effects of changing various independent variables can be evaluated. The paper describes an attempt to use the operant conditioning model as a basis for constructing such a measure of the quality of care provided in residential settings for severely mentally handicapped people. The rationale for adopting this model is described, against a background of other studies in which the same problem has been addressed. The hypothesis, the tasks generated, and the methodological problems encountered, are outlined and discussed.

A subsequent paper will present the results obtained when using the method to measure the quality of care, so defined, in two residential units for severely mentally handicapped children.     

Imagaletics: The application of covert conditioning to athletic performance

Abstract

Imagaletics is the application of covert conditioning to athletic performance. Covert conditioning is a procedure based on operant conditioning and involves the manipulation of imagery. In employing covert conditioning, the individual is asked to imagine the target behavior and then imagine specific consequences to modify that behavior. The assumptions of covert conditioning are presented and the procedures are described with applications to athletic performance.     

Conditioning and post-hospital generalization of nondelusional responses in a chronic psychotic patient

An operant conditioning approach was successful in getting a chronic psychotic patient to give factual answers to direct questions that had previously elicited only delusional responses. Multiple baseline and reversal controls established that the changes were due to the experimental procedure. The subject was a female patient classified as paranoid schizophrenic who had persisted in giving bizarre responses to direct questions regarding her identity, age, and personal history during 26 yr of hospitalization. She was discharged after factual answers to these questions had been obtained, but operant conditioning trials were continued in the community to promote generalization. Two follow-up interviews were conducted 36 and 52 days after discharge to evaluate generalization. No generalization was found in the first interview, but the second gave evidence of some generalization.     

Time-dependent changes in conditioned suppression

Time-dependent changes in a response following aversive conditioning were investigated using a conditioned suppression procedure in a within-subjects design. Four groups of pigeons received Pavlovian conditioning “off the baseline”, immediately followed by an operant task. During the Pavlovian phase, two groups received a forward pairing of a tone with shock, one group received a backward pairing, and one group received a truly random pairing. One of the forward pairing groups also received a delay between the Pavlovian and operant phases. For all groups, key pecking was reinforced on a variable-interval schedule during the operant phase. Testing sessions were identical to training sessions, except that the tone used during Pavlovian conditioning was presented either 0, 15, 30, 45, of 60 minutes after the operant phase began. Testing sessions in which the Pavlovian phase was omitted were also included. The results showed suppression to change as a function of the retention interval, with maximum suppression occurring at intermediate intervals. This U-shaped function was obtained for 11 of the 12 pigeons in the forward-pairing groups and for three of the five in the truly random group. Pigeons in the background pairing group did not show changes in suppression as a function of the retention interval.     

The dynamics of operant conditioning

Existing models of operant learning are relatively insensitive to historical properties of behavior and applicable to only limited data sets. This article proposes a minimal set of principles based on short-term and long-term memory mechanisms that can explain the major static and dynamic properties of operant behavior in both single-choice and multiresponse situations. The critical features of the theory are as follows: (a) The key property of conditioning is assessment of the degree of association between responses and reinforcement and between stimuli and reinforcement; (b) the contingent reinforcement is represented by learning expectancy, which is the combined prediction of response-reinforcement and stimulus-reinforcement associations; (c) the operant response is controlled by the interplay between facilitatory and suppressive variables that integrate differences between expected (long-term) and experienced (short-term) events; and (d) very-long-term effects are encoded by a consolidated memory that is sensitive to the entire reinforcement history. The model predicts the major qualitative features of operant phenomena and then suggests an experimental test of theoretical predictions about the joint effects of reinforcement probability and amount of training on operant choice. We hypothesize that the set of elementary principles that we propose may help resolve the long-standing debate about the fundamental variables controlling operant conditioning.     

Multiple memory stores and operant conditioning: a rationale for memory's complexity

Why does the brain contain more than one memory system? Genetic algorithms can play a role in elucidating this question. Here, model animals were constructed containing a dorsal striatal layer that controlled actions, and a ventral striatal layer that controlled a dopaminergic learning signal. Both layers could gain access to three modeled memory stores, but such access was penalized as energy expenditure. Model animals were then selected on their fitness in simulated operant conditioning tasks. Results suggest that having access to multiple memory stores and their representations is important in learning to regulate dopamine release, as well as in contextual discrimination. For simple operant conditioning, as well as stimulus discrimination, hippocampal compound representations turned out to suffice, a counterintuitive result given findings that hippocampal lesions tend not to affect performance in such tasks. We argue that there is in fact evidence to support a role for compound representations and the hippocampus in even the simplest conditioning tasks.