Defining reinforcess — A problem in communication for consultation in behavior modification

This paper reviews the respondent (Hull-Spence) and operant (Skinnerian) conditioning definitions of reinforcers and reinforcement and demonstrates the need to keep the systems separate when consulting about behavior modification. The two systems are shown to lead to different modification procedures.One important distinction between the systems is whether a reinforcer is simply associated with a response (respondent) or whether it must follow the response (operant). A second important distinction is the definition of negative reinforcement. In respondent conditioning, negative reinforcement entails presenting an aversive stimulus in association with the response and results in a decrease in response rate. In operant conditioning, negative reinforcement entails the removal of an aversive stimulus following a correct response, which results in an increase in response rate.     

The operant control of vocalization in the dog

Control over the vocal responses of three dogs was established using operant-conditioning procedures. Several points of interest were observed in the data. First, fixed-ratio schedules of reinforcement generated a vocal response topography which was similar in detail to that of a “motor” bar-nosing response. Second, vocal responding was brought under the control of external visual stimuli as a result of differential reinforcement. Third, good stimulus control was maintained on a multiple schedule containing a vocal-response component and a bar-response component. Fourth, the stimulus control on the multiple schedule transferred with minimal disruption to a chain schedule requiring a sequence of 10 bar responses followed by 10 vocal responses. Fifth, because vocal and bar responses are not mutually exclusive, concurrent responding tended to develop on the chain schedule.

These results were discussed with reference to the advisability of applying the terms operant and respondent to unconditioned behavior, and, particularly, to unconditioned verbal behavior.

     

Generalization peak shift for autoshaped and operant key pecks.

Pigeons acquired discriminated key pecking between 528- and 540-nm stimuli by either a response-reinforcer (operant group) or a stimulus-reinforcer (autoshaped group) contingency, with other training-schedule parameters comparable over groups. For the birds in the operant group, key pecks intermittently produced grain in the presence of one hue on the key (positive stimulus) but not in the other (negative stimulus). For the birds in the autoshaped group, pecking emerged when grain was intermittently presented independently of key pecking during one key color but was not presented during the other key color. Two independent contingency assays, peck-location comparisons and elimination of differences in reinforcement rate, confirmed the effectiveness of the two training procedures in establishing operant or respondent control of key pecking. After reaching a 10:1, or better, discrimination ratio between key pecks during the two key colors, the birds received a wavelength generalization test. Criterion baseline key-peck rates were comparable for operant and autoshaped groups prior to testing. On the generalization test, performed in extinction, all birds pecked most at a stimulus removed from the positive training stimulus in the direction away from the negative stimulus. In testing, autoshaped "peak" rates (24.5 to 64.9 pecks per minute) were from 33% to 80% higher than rates in the presence of the training stimuli. Respondent peak shift rarely has been reported heretofore, and never this consistently and robustly. These results further confirm the similarity of perceptual processing in classical and operant learning. They are discussed in terms of Spence's gradient-interaction theory and Weiss' (1978) two-process model of stimulus control.     

Selective sensitivity of schedule-induced activity to an operant suppression contingency.

The sensitivity of pigeons' schedule-induced activity to operant consequences was studied in two experiments. During a 30-s interval between food presentations, a keylight stimulus brightened incrementally. Stable terminal key pecking and interim locomotor activity developed. An operant "setback" contingency was applied to activity. The contingency arranged for locomotor movements (detected by a nine-panel floorboard) to be followed by a resetting of the keylight brightness to a dimmer value and a 1-s delay of reinforcement (for individual responses). Experiment 1 showed that activity patterns were highly sensitive to their operant consequences. Accompanying key-peck rates were only transiently affected. In Experiment 2, the setback contingency was imposed during restricted portions of the trial, and differential operant control of activity was demonstrated. However, birds in this study produced higher rates of key pecking as activity rates were reduced. These results suggest that although schedule-induced activity arises in response to the temporal arrangement of stimulus events, this behavior may retain considerable sensitivity to response-consequence relations.     

Stimulus control of respondent and operant key pecking: A single key procedure

Pigeons' responses to a uniformly illuminated response key were either reinforced on a variable-interval one-minute schedule of reinforcement or extinguished for one-minute periods. When 1.5 second signals were presented at the beginning of each component, so as to differentially predict reinforcement, the pigeons pecked at the signals, at rates higher than rates during the remainder of the component. When the brief signals were not differentially predictive of reinforcement, pecking in their presence decreased to near zero levels. Similar results were obtained with signals based upon colors and upon line orientations. Changes in rates of (unreinforced) pecking occurred during the signal whether pigeons responded differentially during the remainder of the component or not. Experiment II demonstrated that the presence of the signal correlated with extinction was not necessary for pecking to develop at the signal which preceded the component in which responding was intermittently reinforced. The experiments demonstrated a clear dissociation of respondent control from operant control of a response. In addition, operant behavior was shown to be relatively insensitive to differing rates of reinforcement, as compared to the sensitivity of respondent behavior to differing rates of reinforcement produced by the very same operant behavior.     

Generalization gradients of inhibition after different amounts of training

Five groups of pigeons received seven sessions of variable-interval reinforcement for pecking a blank white key, followed by either 1, 2, 4, 8, or 16 sessions of training on a successive discrimination in which the positive stimulus was the blank white key and the negative stimulus was a black vertical line on the white key. After training, a generalization test was administered along the line-tilt continuum. Relative gradients of inhibition became steeper with increased amounts of training, and reliably nonhorizontal absolute gradients were obtained only from groups of subjects with at least four days of training. Therefore, inhibitory stimulus control improves with added training. Several problems with the concept of “inhibition” are examined and some implications of the results for theoretical analyses of operant discrimination learning are discussed.     

Theoretical implications of quantitative properties of interval timing and probability estimation in mouse and rat

Although theoretical discussions typically assume that positive and negative reinforcement differ, the literature contains little unambiguous evidence that they produce differential behavioral effects. To test whether the two types of consequences control behavior differently, we pitted money‐gain positive reinforcement and money‐loss‐avoidance negative reinforcement, scheduled through identically programmed variable‐cycle schedules, against each other in concurrent schedules. Contingencies of response‐produced feedback, normally different in positive and negative reinforcement, were made symmetrical. Steeper matching slopes were produced compared to a baseline consisting of all positive reinforcement. This free‐operant differential outcomes effect supports the notion that that stimulus‐presentation positive reinforcement and stimulus‐elimination negative reinforcement are functionally “different.” However, a control experiment showed that the feedback asymmetry of more traditional positive and negative reinforcement schedules also is sufficient to create a “difference” when the type of consequence is held constant. We offer these findings as a small step in meeting the very large challenge of moving negative reinforcement theory beyond decades of relative quiescence.     

Rethinking reinforcement: allocation, induction, and contingency

The concept of reinforcement is at least incomplete and almost certainly incorrect. An alternative way of organizing our understanding of behavior may be built around three concepts: allocation, induction, and correlation. Allocation is the measure of behavior and captures the centrality of choice: All behavior entails choice and consists of choice. Allocation changes as a result of induction and correlation. The term induction covers phenomena such as adjunctive, interim, and terminal behavior-behavior induced in a situation by occurrence of food or another Phylogenetically Important Event (PIE) in that situation. Induction resembles stimulus control in that no one-to-one relation exists between induced behavior and the inducing event. If one allowed that some stimulus control were the result of phylogeny, then induction and stimulus control would be identical, and a PIE would resemble a discriminative stimulus. Much evidence supports the idea that a PIE induces all PIE-related activities. Research also supports the idea that stimuli correlated with PIEs become PIE-related conditional inducers. Contingencies create correlations between "operant" activity (e.g., lever pressing) and PIEs (e.g., food). Once an activity has become PIE-related, the PIE induces it along with other PIE-related activities. Contingencies also constrain possible performances. These constraints specify feedback functions, which explain phenomena such as the higher response rates on ratio schedules in comparison with interval schedules. Allocations that include a lot of operant activity are "selected" only in the sense that they generate more frequent occurrence of the PIE within the constraints of the situation; contingency and induction do the "selecting."     

Stimulus control of Pavlovian facilitation

Two experiments were conducted using an autoshaping procedure with pigeons to examine whether dimensional stimulus control by a Pavlovian facilitator parallels the control established following operant discrimination training. Facilitation training consisted of the presentation of a black vertical line on a white background as the B stimulus in a feature-positive discrimination in which the A stimulus (white keylight) was followed by grain presentation only if preceded by B. In this way, B facilitates or sets the occasion for pecking at A. Subsequent testing for generalization along the line-orientation dimension produced decremental gradients when the facilitation paradigm incorporated an explicit feature-negative stimulus (B−). These results parallel the decremental control obtained following operant discrimination training and suggest that Pavlovian facilitators and instrumental discriminative stimuli are functionally equivalent.     

A selectionist approach to reinforcement.

We describe a principle of reinforcement that draws upon experimental analyses of both behavior and the neurosciences. Some of the implications of this principle for the interpretation of behavior are explored using computer simulations of adaptive neural networks. The simulations indicate that a single reinforcement principle, implemented in a biologically plausible neural network, is competent to produce as its cumulative product networks that can mediate a substantial number of the phenomena generated by respondent and operant contingencies. These include acquisition, extinction, reacquisition, conditioned reinforcement, and stimulus-control phenomena such as blocking and stimulus discrimination. The characteristics of the environment-behavior relations selected by the action of reinforcement on the connectivity of the network are consistent with behavior-analytic formulations: Operants are not elicited but, instead, the network permits them to be guided by the environment. Moreover, the guidance of behavior is context dependent, with the pathways activated by a stimulus determined in part by what other stimuli are acting on the network at that moment. In keeping with a selectionist approach to complexity, the cumulative effects of relatively simple reinforcement processes give promise of simulating the complex behavior of living organisms when acting upon adaptive neural networks.     

Operant control of eye movements

In a monitoring situation eye movements were required in order for signals to be presented. Detection of signals was the reinforcement. A multiple schedule of fixed-interval reinforcement, differential reinforcement of low rate, and fixed-ratio reinforcement was established for eye movements. Results demonstrated that an eye movement can act as an operant controlled by its consequences. Operant control of eye movements has important implications for human factor analysts concerned with "attention".     

Positive conditioned suppression: effects of CS duration

During a brief conditioned stimulus (15 or 30 sec) that terminated with the response-independent delivery of banana pellets, operant responding reinforced by other food pellets according to a variable-interval schedule of reinforcement was suppressed in the squirrel monkey. Conditioned stimuli of longer duration (1, 2, and 3 min) did not reliably affect the rate of operant performance. Brief conditioned stimuli generated homogeneous response patterns of nearly complete suppression. Increasing the CS duration did not enhance responding, as previously reported, but led to alternate bursting and pausing, which suggested a loss of control by the conditioned stimulus. The results suggest that the magnitude of "positive" or "negative" conditioned suppression reflects the strength of the classical conditioning process.     

Incentive processes and the peak shift

Intradimensional operant discrimination schedules were employed, which eliminated the covariation of response and reinforcement rates that are found on most operant baselines. In Phase 1, one keylight (S(1)) controlled an increase in pigeons' treadle pressing, relative to another keylight (S(2)), while being correlated with a decrease in frequency of reinforcement. In Phase 2 both treadle pressing and reinforcement increased in the presence of one keylight, relative to the second. In Phase 1 the relatively flat treadle-press generalization gradients peaked at S(1), whereas the peaks of those in Phase 2 were shifted from S(1) in a direction away from S(2). It was postulated that these positive and negative stimulus-reinforcement contingencies influence the likelihood of obtaining peak shift through the operation of a classically conditioned "central motive state." How response-reinforcement and stimulus-reinforcement contingencies might contribute to the development of inhibitory effects of S(2) is discussed. Autoshaped key pecking also was produced by these procedures. During manipulations of stimuli, the gradients obtained for autoshaped key pecking were narrow and sharply peaked at the food-correlated stimulus (S(2)) in Phase 1. This failure to obtain peak shift for an elicited response suggests a difference in discriminative processes operating in classical and instrumental learning.     

Establishing operations

The first two books on behavior analysis (Skinner, 1938; Keller & Schoenfeld, 1950) had chapter-length coverage of motivation. The next generation of texts also had chapters on the topic, but by the late 1960s it was no longer being given much treatment in the behavior-analytic literature. The present failure to deal with the topic leaves a gap in our understanding of operant functional relations. A partial solution is to reintroduce the concept of the establishing operation, defined as an environmental event, operation, or stimulus condition that affects an organism by momentarily altering (a) the reinforcing effectiveness of other events and (b) the frequency of occurrence of that part of the organism's repertoire relevant to those events as consequences. Discriminative and motivative variables can be distinguished as follows: The former are related to the differential availability of an effective form of reinforcement given a particular type of behavior; the latter are related to the differential reinforcing effectiveness of environmental events. An important distinction can also be made between unconditioned establishing operations (UEOs), such as food deprivation and painful stimulation, and conditioned establishing operations (CEOs) that depend on the learning history of the organism. One type of CEO is a stimulus that has simply been paired with a UEO and as a result may take on some of the motivative properties of that UEO. The warning stimulus in avoidance procedures is another important type of CEO referred to as reflexive because it establishes its own termination as a form of reinforcement and evokes the behavior that has accomplished such termination. Another CEO is closely related to the concept of conditional conditioned reinforcement and is referred to as a transitive CEO, because it establishes some other stimulus as a form of effective reinforcement and evokes the behavior that has produced that other stimulus. The multiple control of human behavior is very common, and is often quite complex. An understanding of unlearned and learned establishing operations can contribute to our ability to identify and control the various components of such multiple determination.     

Discriminative properties of shock-correlated reinforcement in an operant context

A two-choice discrete operant procedure was devised for the study of shock-correlated reinforcement effects in rats. In the presence of one auditory stimulus, responding on one response lever was reinforced with food; with another auditory stimulus, responding on a second lever was reinforced. It was found that discrimination performance of one group, relative to appropriate control groups, was facilitated when electric shock was correlated with reinforcement on one lever and not on the other. Further, relative discrimination levels were found to be higher on the lever correlated with the shock than on the alternate lever. The significance of the results for operant within-S studies and for a mediational theory of shock-correlated reinforcement was discussed.     

Operant and nonoperant vocal responding in the mynah: Complex schedule control and deprivation-induced responding

Several recent studies have been concerned with operant responses that are also affected by nonoperant factors, (e.g., biological constraints, innate behavior patterns, respondent processes). The major reason for studying mynah vocal responding concerned the special relation of avian vocalizations to nonoperant emotional and reflexive systems. The research strategy was to evaluate operant and nonoperant control by comparing the schedule control obtained with the vocal response to that characteristic of the motor responses of other animals. We selected single, multiple, and chain schedules that ordinarily produce disparate response rates at predictable times. In multiple schedules with one component where vocal responding (“Awk”) was reinforced with food (fixed-ratio or fixed-interval schedule) and one where the absence of vocal responding was reinforced (differential reinforcement of other behavior), response rates never exceeded 15 responses per minute, but clear schedule differences developed in response rate and pause time. Nonoperant vocal responding was evident when responding endured across 50 extinction sessions at 25% to 40% of the rate during reinforcement. The “enduring extinction responding” was largely deprivation induced, because the operant-level of naive mynahs under food deprivation was comparable in magnitude, but without deprivation the operant level was much lower. Food deprivation can induce vocal responding, but the relatively precise schedule control indicated that operant contingencies predominate when they are introduced.     

Discriminative effects of massed extincttion

Prior studies have reported that generalization gradients are not steepened if periods of non-reinforcement in S− follow and are not interspersed with periods of reinforcement in S+. Sharper gradients are produced by this massed-extinction procedure if it is preceded by prior discriminative training on a dimension orthogonal to the S+, S− dimension. The present study, using pigeons, found that generalization gradients along the wavelength dimension were steepened by massed-extinction sessions in 570 nm that had been preceded by: (1) discriminative training in which the S+ was a 550-nm light and the S− was a black vertical line superimposed on the 550-nm light; (2) non-differential reinforcement training with a 550-nm light and a black vertical line superimposed on the 550-nm light; (3) reinforcement training with only the 550-nm light. Massed-extinction sessions were administered until the response rate in the presence of the 570-nm stimulus was one-tenth of the mean response rate in the presence of the 550-nm stimulus during prior reinforcement training. Prior studies have used a time-dependent criterion, rather than a response-rate criterion of extinction, and this difference may be responsible for the differences in the effects of massed extinction on stimulus control.     

The establishment of stimulus control by instructions and by differential reinforcement

A repeated acquisition design was used to study the effects of instructions and differential reinforcement on the performance of complex chains by undergraduates. The chains required responding on a series of keys that corresponded to characters that appeared on a monitor. Each day, subjects performed a new chain in a learning session and later relearned the same chain in a test session. Experiment 1 replicated previous research by showing that instructional stimuli paired with the correct responses in the learning sessions, combined with differential reinforcement in both learning and test sessions, resulted in stimulus control by the characters in each link. Experiment 2 separated the effects of instructional stimuli and differential reinforcement, and showed that stimulus control by the characters could be established solely by differential reinforcement during the test sessions. Experiment 3 showed that when a rule specified the relation between learning and test sessions, some subjects performed accurately in the test sessions without exposure to any differential consequences. This rule apparently altered the stimulus control properties of the characters much as did differential reinforcement during testing. However, compared to differential reinforcement, the rule established stimulus control more quickly.     

Conditioned suppression of an avoidance response by a stimulus paired with food

Three food-deprived Long-Evans rats were exposed to a non-discriminated shock avoidance procedure. Superimposed upon this operant avoidance baseline were periodic presentations of a conditioned stimulus that was paired with food, the unconditioned stimulus. These pairings resulted in increases in the rate of shock over that recorded when the conditioned stimulus was not present. A traditional suppression ratio failed to reveal any differential effect of the conditioned stimulus on the overall rate of avoidance responding, although all subjects showed a consistent pattern of pausing and postshock response bursts during presentations of the conditioned stimulus. When food was withheld during a final extinction phase, the conditioned stimulus ceased to occasion increases in shock rates and disruptive postshock response bursts were eliminated. An analysis of conditioned suppression procedures is proposed that stresses not only operant-Pavlovian or appetitive-aversive incompatibility, but also the manner in which the baseline schedule of reinforcement affects operant behavior changes that are elicited by the superimposed Pavlovian procedure.