In vitro reinforcement of hippocampal bursting: a search for Skinner's atoms of behavior.

A novel "in vitro reinforcement" paradigm was used to investigate Skinner's (1953) hypotheses (a) that operant behavior is made up of infinitesimal "response elements" or "behavioral atoms" and (b) that these very small units, and not whole responses, are the functional units of reinforcement. Our tests are based on the assumption that behavioral atoms may plausibly be represented at the neural level by individual cellular responses. As a first approach, we attempted to reinforce the bursting responses of hippocampal units in a highly reduced brain-slice preparation with local micropressure applications of behaviorally reinforcing dopaminergic drugs. The same injections were administered independently of bursting to provide a "noncontingent" control for nonspecific stimulation or facilitation of firing. It was found that the bursting responses of individual CA1 pyramidal neurons may be progressively facilitated in a dose-related manner by response-contingent (but not noncontingent) injections of dopamine itself, the dopamine D1-preferring agonist SKF 82958, the D3-preferring agonist quinpirole, and the D2-like selective agonist (+)-4-propyl-9 hydroxynapthoxazine. These findings support the conclusion that unit bursting responses can be reinforced in vitro in hippocampal slices, and they further suggest that the same dopamine receptor subtypes are involved in both cellular and behavioral operant conditioning. The results thus provide indirect support for Skinner's atoms-of-behavior hypothesis.     

Evaluation of infant vocal conditioning.

The use of the operant conditioning paradigm, as it has been applied to infant social, vocal behavior, reflects a failure to take into account the social nature of human infants over and above the rigid theoretical rationale of the paradigm. It is argued that: (1) the baseline procedure used in operant conditioning studies is methodologically and conceptually invalid; (2) the reinforcing stimulus used in social conditioning studies elicits the very response that it is assigned to reinforce; and (3) the effect of the response-reinforcer relationship is not the reinforcement (strengthening) of vocal response rate.     

Occasional reinforced responses during extinction can slow the rate of reacquisition of an operant response

Three experiments with rats examined reacquisition of an operant response after either extinction or a response-elimination procedure that included occasional reinforced responses during extinction. In each experiment, reacquisition was slower when response elimination had included occasional reinforced responses, although the effect was especially evident when responding was examined immediately following each response-reinforcer pairing during reacquisition (Experiments 2 and 3). An extinction procedure with added noncontingent reinforcers also slowed reacquisition (Experiment 3). The results are consistent with research in classical conditioning (Bouton, M. E., Woods, A. M., & Pineño, O. (2004). Occasional reinforced trials during extinction can slow the rate of rapid reacquisition. Learning & Motivation, 35, 371-390) and suggest that rapid reacquisition after extinction is analogous to a renewal effect that occurs when reinforced responses signal a return to the conditioning context. Clinical implications are also discussed.     

Dopamine transporter blockade increases LTP in the CA1 region of the rat hippocampus via activation of the D3 dopamine receptor

Dopamine has been demonstrated to be involved in the modulation of long-term potentiation (LTP) in the CA1 region of the hippocampus. As monoamine transporter blockade will increase the actions of endogenous monoamine neurotransmitters, the effect of a dopamine transporter (DAT) antagonist on LTP was assessed using field excitatory postsynaptic potentials recorded in the CA1 region of the rat hippocampal slice preparation. Application of the DAT-specific blocker GBR 12,935 produced a significant enhancement in LTP of Schaffer collateral synapses in the CA1 at concentrations as low as 100 nM. A selective D1/D5 dopamine receptor antagonist (SCH 23,390, 1 microM) did not affect the ability of GBR 12,935 to enhance LTP, whereas application of the D3 dopamine receptor antagonist U 99,194 (1 microM) blocked the GBR 12,935-induced enhancement in LTP. In addition, a D3 dopamine receptor agonist (7-OH-DPAT, 1 microM) caused a significant increase in LTP, an effect that was also blocked by U 99,194 (3 microM). These results suggest that either endogenously released dopamine (facilitated by DAT blockade) or exogenously applied dopamine agonist can act to increase LTP in the CA1 of the hippocampus via activation of the D3 subtype of dopamine receptor.     

Tactical contingencies in the experimental analysis of reinforcement and operant classes

In his “Tactics of Scientific Research” (1960), his work on avoidance, his discovery of equivalence classes and his cautions on applications of coercion, Murray Sidman created high standards for behavior analytic research. I illustrate his influence in the context of three examples he might have characterized as pilot studies. Each examined trial N+1 response probabilities depending on whether trial N responding had produced a reinforcer. Differentially reinforced interresponse times, keys pecked in arbitrary matching, and two-key response sequences provided no robust evidence that reinforcing some response property on trial N raises the probability of responding with that property on trial N+1. These negative findings shed light on the nature of operant classes and on the relation of reinforcers to the responses that produced them. Through selection, reinforcers create operant classes and engender variations of the responses within those classes; operant classes are held together by common contingencies. Sidman extended our understanding of operant classes by expanding them to include equivalence relations.     

Neural events in the reinforcement contingency

When neural events are analyzed as stimuli and responses, functional relations among them and among overt stimuli and responses can be unveiled. The integration of neuroscience and the experimental analysis of behavior is beginning to provide empirical evidence of involvement of neural events in the three-term contingency relating discriminative stimuli, responses, and consequences. This paper is aimed at highlighting exemplar instances in the development of this issue. It has long been known that the electrical stimulation of certain cerebral areas can have a reinforcing function. Extraordinary technological advances in recent years show that neural activity can be selected by consequences. For example, the activity of in vitro isolated neurons that receive dopamine as a reinforcer functions as a cellular analogue of operant conditioning. The in vivo activity of populations of neurons of rats and monkeys can be recorded on an instant-to-instant basis and can then be used to move mechanical arms or track a target as a function of consequences. Neural stimulation acts as a discriminative stimulus for operant responses that are in turn maintained by neural consequences. Together with investigations on the molecular basis of classical conditioning, those studies are examples of possibilities that are being created for the study of behavior-environment interactions within the organism. More important, they show that, as an element in the three-term contingency, neural activity follows the same laws as other events.     

Facilitation of discriminated lever-press avoidance by noncontingent shocks

Two experiments investigated the effect of presenting a brief noncontingent, unsignaled shock prior to warning-signal onset on discriminated lever-press avoidance behavior in rats. The main finding was that noncontingent shock delivered 2 sec but not 16 sec prior to warning-signal onset facilitated avoidance performance and learning. These results were predicted on the basis of shock-elicited response bursting which was expected to increase the operant level of the lever-press response at short shock-warning signal intervals. It was suggested that species-specific limitations on rats' lever-press avoidance are not unconditional, but rather can be overcome with environmental manipulations.     

Bidirectional operant conditioning of heart rate in rats with sucrose reward

Sucrose was used to reinforce heart rate (HR) increases in one group of unrestrained rats and to reinforce decreases in another. A third group received noncontingent sucrose presentations, and a fourth was presented with an empty dipper. After 10 conditioning sessions HR, as measured by changes in tonic levels, was significantly greater in the Fast group than in the Noncontingent group, which in turn maintained significantly higher levels than the Slow group. There was no difference between the Noncontingent and No-sucrose groups. Behavioral observations indicated that the Fast animals increased the percentage of time spent in such high-HR categories as rearing and walking; however, there were no corresponding systematic decreases in the Slow animals. Moreover, the Fast and Slow groups differed significantly with regard to the HRs associated with five behavioral categories. The overall pattern of results suggests that some degree of cardiospecific effect may have been exerted by the operant contingencies.     

Effects of delayed reinforcement on infant vocalization rate.

Three previous studies have failed to demonstrate conditioning in infants using a 3-s delay of reinforcement. The effects of a delayed reinforcement schedule on vocalization rates therefore were explored in a single-subject repeated-reversal experimental design for 3 4- to 6-month-old normally developing infants. Each infant received delayed social reinforcement from his or her parent for vocalizing. The comparison condition was a schedule of differential reinforcement of behavior other than vocalizations to control for elicitation by social stimulation. An operant level of infant vocalizations was the initial condition, after which the differential reinforcement schedule was implemented in an across-subjects multiple baseline design. Infants' vocalization rates increased above levels measured during differential reinforcement following onset of the delayed reinforcement condition. Also, vocalization rates decreased during differential reinforcement compared to operant levels. The successful use of delayed reinforcement schedules with infants in this study, as opposed to others, is discussed in terms of procedural differences among them.     

Operant conditioning of a small-scale muscle response

Six naive male subjects were reinforced for responses in the 20-30 microvolt range, and two each for those in 10-20 and 30-40 microvolt ranges. Records were taken of 15 min of "settling down," 15 min of initial operant level responding, 30 min of conditioning, and 45 min of extinction, 30 min with light present and 15 min without. The results were: (1) small-scale muscle potentials from the thenar eminence, in the 10-20, 20-30, and 30-40 microvolt ranges, were conditioned, using pointer movement as the reinforcement; (2) the response rate in adjacent ranges of greater and lesser amplitude also increased during such conditioning and decreased during the subsequent extinction; (3) during conditioning the frequency of response distribution shifted toward the reinforced range; (4) subjects were unaware of the reinforced response; (5) observing a noncontingent moving pointer increased the response rate in the 20-30 microvolt range, but did not lead to conditioning.     

Noncontingent reinforcement competes with response performance

Noncontingent reinforcement is a commonly used procedure to decrease levels of problem behavior. Goals of this intervention are to decrease motivation, responding, and the functional relation between behavior and consequences, but it could also possibly compete with performance of alternative desirable responses. In the current study, we assessed the effects of noncontingent reinforcement arranged from 0% to 100% of sessions on performance of alternative responding across two experiments. Experiment 1 assessed manding (i.e., requests) maintained by attention and tangibles with a child with developmental disabilities and Experiment 2 assessed keypecking maintained by food with six pigeons. We extended previous research by (a) showing that noncontingent reinforcement competes with both the acquisition and maintenance (performance) of an alternative response, (b) extending the generality of the findings across nonhuman and human participants, and (c) eliminating influence of sequence effects through random manipulations of noncontingent value in pigeons. Overall, greater amounts of noncontingent reinforcement competed with both acquisition and maintenance of alternative responding.     

Systemic blockade of D2-like dopamine receptors facilitates extinction of conditioned fear in mice

Extinction of conditioned fear in animals is the explicit model of behavior therapy for human anxiety disorders, including panic disorder, obsessive-compulsive disorder, and post-traumatic stress disorder. Based on previous data indicating that fear extinction in rats is blocked by quinpirole, an agonist of dopamine D2 receptors, we hypothesized that blockade of D2 receptors might facilitate extinction in mice, while agonists should block extinction, as they do in rats. One day after fear conditioning mice with three pairings of a white noise conditional stimulus (CS) with moderate footshock, we injected the D2 antagonist, sulpiride, the D2 agonist, quinpirole, or vehicle, just before repeated CS presentations to generate extinction. We assayed fear by measuring behavioral freezing during extinction presentations and then drug-free during CS presentations 1 d later. We found that sulpiride injections before extinction training facilitated extinction memory 24 h later, while quinpirole partially blocked extinction memory compared with vehicle-injected controls. Notably, sulpiride treatment yielded significant extinction after spaced CS presentations, which yield no extinction at all in vehicle-treated mice. These findings suggest that dopamine D2-mediated signaling contributes physiological inhibition of extinction, and that D2 antagonists may be useful adjuncts to behavior therapy of human anxiety disorders.     

The efficacy of auditory and visual conjugate reinforcers in infant conditioning

The effectiveness of the separate auditory and visual components of a combined auditory-visual conjugate reinforcer was assessed in 273-month-old infants. All conditions of contingent reinforcement were equally effective in promoting acquisition, but only the removal of the singly presented auditory component resulted in significant response diminution. Although noncontingent stimulation in all conditions was ineffective in altering operant footkicks, it did sustain visual attention at a high level equivalent to that produced by contingent stimulation over the major portion of acquisition. The results were discussed in terms of the reinforcing properties of varied stimulation.     

Localized intracaudate dopamine D2 receptor activation during the post-training period improves memory for visual or olfactory conditioned emotional responses in rats.

Rats with cannulas aimed at the posteroventral (PV) or ventrolateral (VL) areas of the caudate nucleus were trained on a conditioned emotional response (CER) task. Post-training microinjections of the indirect catecholamine agonist, d-amphetamine (5 micrograms), or of the dopamine D2 receptor agonist, LY171555 (1 microgram), into the PV area improved retention of a CER with a visual CS, but had no effect on a CER with an olfactory CS. Post-training injections of the same two drugs into the VL area improved retention of a CER with an olfactory CS, but had no effect on a CER with a visual CS. Post-training injections of the dopamine D1 receptor agonist, SKF38393 (0.5, 1.0, 2.0 micrograms), into either site had no effects on either CER. These findings suggest that different areas of the caudate nucleus mediate acquisition of CERs with different CSs, possibly implicating the topographically organized corticostriatal innervation in the acquisition of certain types of memories in the caudate nucleus. The findings also suggest that dopamine D2 receptors in the caudate nucleus are involved in the acquisition of these CERs.     

Response-outcome contingency: Behavioral and judgmental effects of appetitive and aversive outcomes with college students

In two experiments, positive, negative, and zero response-outcome contingencies were responded to and rated by college students under a free-operant procedure. In Experiment 1, outcomes were either neutral or were associated with point gain. In Experiment 2, subjects were administered different outcome treatments: neutral outcomes, outcomes associated with money gain, or outcomes associated with money loss. In both experiments, subjects' judgments of response-outcome contingency and their operant responses were each strong linear functions of ΔP, the difference between the probability of an outcome given a response and the probability of an outcome given no response. Appetitive and aversive outcomes produced opposite and symmetrical response patterns. In Experiment 1, no differences in ratings occurred with neutral or appetitive outcomes; however, in Experiment 2, more potent appetitve outcomes led to somewhat more extreme ratings than either neutral or aversive outcomes. Increasing outcome probability produced only a slight bias in ratings of noncontingent problems in Experiment 1 and no bias in Experiment 2. Contrary to predictions derived from an analysis of superstitious behavior, increasing outcome probability in noncontingent problems decreased operant responding when outcomes were appetitive and increased operant responding when outcomes were aversive. Trend analyses revealed that Δ P was superior to several other metrics in predicting subjects' estimates of contingency and the behavioral effects of contingency. Operant responding was in closer accord with matching predictions than with maximizing predictions.     

Conditioned fear is modulated by D2 receptor pathway connecting the ventral tegmental area and basolateral amygdala

Excitation of the mesocorticolimbic pathway, originating from dopaminergic neurons in the ventral tegmental area (VTA), may be important for the development of exaggerated fear responding. Among the forebrain regions innervated by this pathway, the amygdala is an essential component of the neural circuitry of conditioned fear. The functional role of the dopaminergic pathway connecting the VTA to the basolateral amygdala (BLA) in fear and anxiety has received little attention. In vivo microdialysis was performed to measure dopamine levels in the BLA of Wistar rats that received the dopamine D(2) agonist quinpirole (1 μg/0.2 μl) into the VTA and were subjected to a fear conditioning test using a light as the conditioned stimulus (CS). The effects of intra-BLA injections of the D(1) antagonist SCH 23390 (1 and 2 μg/0.2 μl) and D(2) antagonist sulpiride (1 and 2 μg/0.2 μl) on fear-potentiated startle (FPS) to a light-CS were also assessed. Locomotor performance was evaluated by use of open-field and rotarod tests. Freezing and increased dopamine levels in the BLA in response to the CS were both inhibited by intra-VTA quinpirole. Whereas intra-BLA SCH 23390 did not affect FPS, intra-BLA sulpiride (2 μg) inhibited FPS. Sulpiride's ability to decrease FPS cannot be attributed to nonspecific effects because this drug did not affect motor performance. These findings indicate that the dopamine D(2) receptor pathway connecting the ventral tegmental area and the basolateral amygdala modulates fear and anxiety and may be a novel pharmacological target for the treatment of anxiety.     

The emergence of conditioned reinforcement from observation

We report an experiment in which observations of peers by six 3-5-year-old participants under specific conditions functioned to convert a small plastic disc or, for one participant, a small piece of string, from a nonreinforcer to a reinforcer. Prior to the observational procedure, we compared each participant's responding on (a) previously acquired performance tasks in which the child received either a preferred food item or the disc (string) for correct responses, and (b) the acquisition of new repertoires in which the disc (string) was the consequence for correct responses. Verbal corrections followed incorrect responses in the latter tasks. The results showed that discs and strings did not reinforce correct responses in the performance tasks, but the food items did; nor did the discs and strings reinforce correct responses in learning new repertoires. We then introduced the peer observation condition in which participants engaged in a different performance task in the presence of a peer who also performed the task. A partition blocked the participants from seeing the peers' performance. However, participants could observe peers receiving discs or strings. Participants did not receive discs or strings regardless of their performance. Peer observation continued until the participants either requested discs or strings repeatedly, or attempted to take discs or strings from the peers. Following the peer observation condition, the same performance and acquisition tasks in which participants had engaged prior to observation were repeated. The results showed that the discs and strings now reinforced correct responding for both performance and acquisition for all participants. We discuss the results with reference to research involving nonhuman subjects that demonstrated the observational conditioning of reinforcers.     

Operant techniques used in stuttering therapy: A review

This article reviews research concerning the use of operant conditioning in stuttering therapy, and discusses the clinical implications of this literature. In order to be considered for this review, a report had to treat the findings of research specifically designed to use operant conditioning in the manipulation of speech disfluency. This body of experimental literature clearly indicates that operant management techniques can effectively reduce stuttering with punishment of disfluencies producing more notable results than reinforcement of fluent responses. Operant conditioning is an effective means of modifying stuttering behavior and should be more visible in terms of procedures used by speech therapists.     

Sexual reinforcement in the female rat.

Sexual reinforcement in the female rat was studied in a preparation that allowed continuous operant responding for access to a male rat leading to intromission. Experiment 1 used a high operant level nose-poke response to test the possible reinforcing effects of some components of access to a male. A simple tone stimulus used as a conditioned reinforcer and two odor stimuli, target male bedding and emulsified preputial gland, were tested. None of these contingent events altered responding above or below operant level. Access to the male, which was always accompanied by intromission, immediately increased response rate when it was made contingent upon the nose-poke response. Performance on fixed-ratio schedules was erratic, and response rate was low in comparison to typical food-reinforced responding. An interresponse-time analysis indicated, however, that some effect of the ratio contingency may have been present. In Experiment 2, several modifications of the procedure were tested with the objective of creating a more tractable preparation for behavior analysis. Response type and the hormone delivery method were changed, and 2 target males were used instead of 1. The latter tripled the average number of reinforcers earned in a single session. Differences between sexual and other reinforcers are discussed in terms of procedural, quantitative, and motivational aspects of the sexual reinforcement procedure.     

A comparison of the key-peck and treadle-press operants in the pigeon: differential-reinforcement-of-low-rate schedule of reinforcement

Key pecking and treadle pressing in pigeons were compared under concurrent (key-treadle) and single-operant differential-reinforcement-of-low-rate schedules of food reinforcement ranging from 5 to 60 sec (concurrent procedure) or 5 to 120 sec (single-operant procedure). Under both procedures, the two operants followed the same general law: decreasing response rate and reinforcement rate and increasing number of responses per reinforcement as a function of increasing schedule interval. High correlations were found between key pecking and treadle pressing for the measures of response rate, reinforcement rate, and responses per reinforcement. Regression equations allowed the prediction of treadle pressing from key pecking. More bursting occurred in responding to the key, and key pecking showed a more precise temporal discrimination than treadle pressing. A test for sequential dependencies between key and treadle responses showed significant dependencies not only under the concurrent procedure but also in data created artificially by merging key and treadle sequences from different pigeons under the concurrent procedure and from the same pigeon under the single-operant procedure. It seems likely that the sequential dependencies found were due to the independent action of the schedule on each operant and that behavioral dependencies did not occur with the concurrent training procedure. The key-peck operant does not appear to have any special qualities that preclude its use in discovering general laws of behavior, at least under the differential-reinforcement-of-low-rate schedule. The usefulness of the key peck in other situations requires direct experimental study.