Effects of instructions and reinforcement-feedback on human operant behavior maintained by fixed-interval reinforcement

In three experiments, human subjects were trained on a five-component multiple schedule with different fixed intervals of monetary reinforcement scheduled in the different components. Subjects uninstructed about the fixed-interval schedules manifested high and generally equivalent rates regardless of the particular component. By comparison, subjects given instructions about the schedules showed orderly progressions of rates and temporal patterning as a function of the interreinforcement intervals, particularly when feedback about reinforcement was delivered but also when reinforcement-feedback was withheld. Administration of the instructions-reinforcement combination to subjects who had already developed poorly differentiated behavior, however, did not make their behavior substantially better differentiated. When cost was imposed for responding, both instructed and uninstructed subjects showed low and differentiated rates regardless of their prior histories. It was concluded that instructions can have major influences on the establishment and maintenance of human operant behavior.     

Making sense of sensitivity in the human operant literature

Human operant behavior is often said to be controlled by different variables or governed by different processes than nonhuman operant behavior. Support for this claim within the operant literature comes from data suggesting that human behavior is often insensitive to schedules of reinforcement to which nonhuman behavior has been sensitive. The data that evoke the use of the terms sensitivity and insensitivity, however, result from both between-species and within-subject comparisons. We argue that because sensitivity is synonymous with experimental control, conclusions about sensitivity are best demonstrated through within-subject comparisons. Further, we argue that even when sensitivity is assessed using within-subject comparisons of performance on different schedules of reinforcement, procedural differences between studies of different species may affect schedule performance in important ways. We extend this argument to age differences as well. We conclude that differences across populations are an occasion for more precise experimental analyses and that it is premature to conclude that human behavior is controlled by different processes than nonhuman behavior.     

Performances on ratio and interval schedules of reinforcement: Data and theory

Two differences between ratio and interval performance are well known: (a) Higher rates occur on ratio schedules, and (b) ratio schedules are unable to maintain responding at low rates of reinforcement (ratio “strain”). A third phenomenon, a downturn in response rate at the highest rates of reinforcement, is well documented for ratio schedules and is predicted for interval schedules. Pigeons were exposed to multiple variable-ratio variable-interval schedules in which the intervals generated in the variable-ratio component were programmed in the variable-interval component, thereby “yoking” or approximately matching reinforcement in the two components. The full range of ratio performances was studied, from strained to continuous reinforcement. In addition to the expected phenomena, a new phenomenon was observed: an upturn in variable-interval response rate in the midrange of rates of reinforcement that brought response rates on the two schedules to equality before the downturn at the highest rates of reinforcement. When the average response rate was corrected by eliminating pausing after reinforcement, the downturn in response rate vanished, leaving a strictly monotonic performance curve. This apparent functional independence of the postreinforcement pause and the qualitative shift in response implied by the upturn in variable-interval response rate suggest that theoretical accounts will require thinking of behavior as partitioned among at least three categories, and probably four: postreinforcement activity, other unprogrammed activity, ratio-typical operant behavior, and interval-typical operant behavior.     

Theoretical implications of quantitative properties of interval timing and probability estimation in mouse and rat

Although theoretical discussions typically assume that positive and negative reinforcement differ, the literature contains little unambiguous evidence that they produce differential behavioral effects. To test whether the two types of consequences control behavior differently, we pitted money‐gain positive reinforcement and money‐loss‐avoidance negative reinforcement, scheduled through identically programmed variable‐cycle schedules, against each other in concurrent schedules. Contingencies of response‐produced feedback, normally different in positive and negative reinforcement, were made symmetrical. Steeper matching slopes were produced compared to a baseline consisting of all positive reinforcement. This free‐operant differential outcomes effect supports the notion that that stimulus‐presentation positive reinforcement and stimulus‐elimination negative reinforcement are functionally “different.” However, a control experiment showed that the feedback asymmetry of more traditional positive and negative reinforcement schedules also is sufficient to create a “difference” when the type of consequence is held constant. We offer these findings as a small step in meeting the very large challenge of moving negative reinforcement theory beyond decades of relative quiescence.     

Operant conditioning in redwinged blackbirds

An operant conditioning technique for use with passerine birds is described. Two redwinged blackbirds were successfully conditioned to perch-hop for food reinforcement. Continuous reinforcement and fixed-ratio schedules involving substantial ratio requirements were used to maintain this response. The behavior of the two redwinged blackbirds was comparable to that of more conventional organisms working on similar schedules of reinforcement.     

Conditioned reinforcement: Experimental and theoretical issues

The concept of conditioned reinforcement has received decreased attention in learning textbooks over the past decade, in part because of criticisms of its validity by major behavior theorists and in part because its explanatory function in a variety of different conditioning procedures has become uncertain. Critical data from the major procedures that have been used to investigate the concept (second-order schedules, chain schedules, concurrent chains, observing responses, delay-of-reinforcement procedures) are reviewed, along with the major issues of interpretation. Although the role played by conditioned reinforcement in some procedures remains unresolved, the results taken together leave little doubt that the underlying idea of conditioned value is a critical component of behavior theory that is necessary to explain many different types of data. Other processes (marking, bridging) may also operate to produce effects similar to those of conditioned reinforcement, but these clearly cannot explain the full domain of experimental effects ascribed to conditioned reinforcement and should be regarded as complements to the concept rather than theoretical competitors. Examples of practical and theoretical applications of the concept of conditioned reinforcement are also considered.     

Control rate of response or reinforcement and amphetamine's effect on behavior.

The roles of control response rate and reinforcement frequency in producing amphetamine's effect on operant behavior were evaluated independently in rats. Two multiple schedules were arranged in which one variable, either response rate or reinforcement frequency, was held constant and the other variable manipulated. A multiple differential-reinforcement-of-low-rate seven-second yoked variable-interval schedule was used to equate reinforcement frequencies at different control response rates between multiple-schedule components. Amphetamine increased responding under the variable-interval component. In contrast, amphetamine decreased responding equivalently between components of a multiple random-ratio schedule that produced similar control response rates at different reinforcement frequencies. The results provide experimental support to the rate-dependency principle that control rate of responding is an important determinant of amphetamine's effect on operant behavior.     

Stimulus Fading and Response Elaboration in Differential Reinforcement for Alternative Behavior

A hallmark of applied behavior analysis is the development of function‐based interventions for problem behavior. A widely recommended function‐based intervention is differential reinforcement of alternative behavior (DRA), in which reinforcement is contingent upon socially acceptable alternatives to problem behavior (e.g., teaching communication skills). Typically, DRA is introduced under rich schedules of reinforcement. Although effective for initiating behavior change, rich schedules are often impractical in the natural setting. In this study, we evaluated the extent to which a stimulus fading program could be employed to elaborate alternative behavior (mands) in two individuals diagnosed with an autism spectrum disorder. For both participants, problem behavior was reduced substantially upon implementation of the DRA procedure. Further, problem behavior rates remained low and mand rates decreased to more practical levels as the DRA behavioral requirements increased during the fading program. The fading approach demonstrated in this paper may be a useful component of intervention packages for clinicians. Copyright © 2015 John Wiley & Sons, Ltd.     

APPLIED IMPLICATIONS OF REINFORCEMENT HISTORY EFFECTS

Although the influence of reinforcement history is a theoretical focus of behavior analysis, the specific behavioral effects of reinforcement history have received relatively little attention in applied research and practice. We examined the potential effects of reinforcement history by reviewing nonhuman, human operant, and applied research and interpreted the findings in relation to possible applied significance. The focus is on reinforcement history effects in the context of reinforcement schedules commonly used either to strengthen behavior (e.g., interval schedules) or commonly used to decrease behavior (e.g., extinction).     

A comparison of dense-to-lean and fixed lean schedules of alternative reinforcement and extinction

Behavior-reduction interventions typically employ dense schedules of alternative reinforcement in conjunction with operant extinction for problem behavior. After problem behavior is reduced in the initial treatment stages, schedule thinning is routinely conducted to make the intervention more practical in natural environments. In the current investigation, two methods for thinning alternative reinforcement schedules were compared for 3 clients who exhibited severe problem behavior. In the dense-to-lean (DTL) condition, reinforcement was delivered on relatively dense schedules (using noncontingent reinforcement for 1 participant and functional communication training for 2 participants), followed by systematic schedule thinning to progressively leaner schedules. During the fixed lean (FL) condition, reinforcement was delivered on lean schedules (equivalent to the terminal schedule of the DTL condition). The FL condition produced a quicker attainment of individual treatment goals for 2 of the 3 participants. The results are discussed in terms of the potential utility of using relatively lean schedules at treatment outset.     

Induced attack during fixed-ratio and matched-time schedules of food presentation

Adjunctive or induced behavior is generated during a variety of schedules of reinforcement. Several theoretical conceptualizations suggest that rate of reinforcement is the primary variable controlling the strength or levels of induced behavior. The operant response requirement within the schedule context has not been extensively studied as a determinant of induced responding. In the present study, levels of induced attack by food-deprived pigeons against restrained conspecifics were compared during response-dependent and response-independent schedules of food presentation equated or yoked interval-by-interval for reinforcement frequency. Experiment 1 compared levels of attack induced by fixed-ratio schedules of key pecking and yoked "matched-time" schedules. Experiment 2 similarly compared chained fixed-ratio 1 fixed-ratio 74 and yoked chained matched-time matched-time schedules. In both experiments, the response-dependent schedules generated greater levels (amount and probability) of induced attack than the response-independent time-based schedules. Thus, the ratio response requirement may be an important determinant of levels of induced responding, and the lower levels of attack observed during the response-independent condition may not be due to the absence of stimuli predicting food presentations. It is concluded that rate of reinforcement is not the sole variable determining levels of induced responding and that response-based and time-based schedules differ in their generation of induced responding.     

Schedule control of the vocal behavior of Cebus monkeys

The vocal behavior of three Cebus monkeys was maintained by fixed-ratio schedules of response dependent reinforcement at values between fixed-ratio 1 and fixed-ratio 15. In one monkey that was exposed to variable-interval, fixed-interval, and conjunctive fixed-ratio fixed-interval schedules of reinforcement, vocal responding occurred at a low rate, but schedule-appropriate patterns were maintained. The rates and patterns of responding engendered indicated that the vocal operant can be brought under schedule control in the monkey by the use of response-dependent reinforcement.     

Concurrent schedules of positive and negative reinforcement: differential-impact and differential-outcomes hypotheses

Considerable evidence from outside of operant psychology suggests that aversive events exert greater influence over behavior than equal-sized positive-reinforcement events. Operant theory is largely moot on this point, and most operant research is uninformative because of a scaling problem that prevents aversive events and those based on positive reinforcement from being directly compared. In the present investigation, humans' mouse-click responses were maintained on similarly structured, concurrent schedules of positive (money gain) and negative (avoidance of money loss) reinforcement. Because gains and losses were of equal magnitude, according to the analytical conventions of the generalized matching law, bias (log b (double dagger) 0) would indicate differential impact by one type of consequence; however, no systematic bias was observed. Further research is needed to reconcile this outcome with apparently robust findings in other literatures of superior behavior control by aversive events. In an incidental finding, the linear function relating log behavior ratio and log reinforcement ratio was steeper for concurrent negative and positive reinforcement than for control conditions involving concurrent positive reinforcement. This may represent the first empirical confirmation of a free-operant differential-outcomes effect predicted by contingency-discriminability theories of choice.     

Operant and nonoperant vocal responding in the mynah: Complex schedule control and deprivation-induced responding

Several recent studies have been concerned with operant responses that are also affected by nonoperant factors, (e.g., biological constraints, innate behavior patterns, respondent processes). The major reason for studying mynah vocal responding concerned the special relation of avian vocalizations to nonoperant emotional and reflexive systems. The research strategy was to evaluate operant and nonoperant control by comparing the schedule control obtained with the vocal response to that characteristic of the motor responses of other animals. We selected single, multiple, and chain schedules that ordinarily produce disparate response rates at predictable times. In multiple schedules with one component where vocal responding (“Awk”) was reinforced with food (fixed-ratio or fixed-interval schedule) and one where the absence of vocal responding was reinforced (differential reinforcement of other behavior), response rates never exceeded 15 responses per minute, but clear schedule differences developed in response rate and pause time. Nonoperant vocal responding was evident when responding endured across 50 extinction sessions at 25% to 40% of the rate during reinforcement. The “enduring extinction responding” was largely deprivation induced, because the operant-level of naive mynahs under food deprivation was comparable in magnitude, but without deprivation the operant level was much lower. Food deprivation can induce vocal responding, but the relatively precise schedule control indicated that operant contingencies predominate when they are introduced.     

Response rate and sensitivity to the molar feedback function relating response and reinforcement rate on VI+ schedules of reinforcement

In 5 experiments, the author examined rats' sensitivity to the molar feedback function relating response rate to reinforcement rate on schedules of reinforcement. These studies demonstrated that, at lower rates of responding, rats' performance on variable ratio (VR), variable interval (VI), and variable interval with linear feedback loop (VI+) schedules was determined largely by reinforcement of interresponse times; response rates were faster on VR than on both VI and VI+ schedules. In contrast, when procedures were adopted to maintain high rates of response, rats showed sensitivity to the molar characteristics of the schedules; they responded as fast on a VI+ schedule as on a VR schedule and faster on both of these schedules than on a yoked VI schedule. When the variance of response rate was manipulated, this factor was noted as an important element in determining sensitivity to the molar characteristics of the schedule.     

The influence of upcoming food-pellet delivery on subjects' responding for 1% sucrose reinforcement delivered by concurrent random-interval schedules

Researchers have demonstrated that rats' rates of operant responding that are maintained by 1% liquid-sucrose reinforcement will increase if food-pellet reinforcement is upcoming within the same session. The authors investigated whether a similar induction effect would be observed when rats pressed a lever for 1% sucrose that was delivered by concurrent random-interval schedules of reinforcement. Results demonstrated that upcoming noncontingent food-pellet delivery increased absolute response rates on the concurrent schedules in 10 of 12 possible instances. Upcoming food-pellet delivery also increased subjects' sensitivity to reinforcement on the concurrent schedules, as measured by the generalized matching law (W. M. Baum, 1974), in 5 of 6 possible instances. The present results extended the finding of induction to responding on concurrent schedules. They also added to evidence suggesting that the effect occurs because the reinforcing value of the weak reinforcer (i.e., the 1% sucrose) has been increased.     

The dynamics of operant conditioning

Existing models of operant learning are relatively insensitive to historical properties of behavior and applicable to only limited data sets. This article proposes a minimal set of principles based on short-term and long-term memory mechanisms that can explain the major static and dynamic properties of operant behavior in both single-choice and multiresponse situations. The critical features of the theory are as follows: (a) The key property of conditioning is assessment of the degree of association between responses and reinforcement and between stimuli and reinforcement; (b) the contingent reinforcement is represented by learning expectancy, which is the combined prediction of response-reinforcement and stimulus-reinforcement associations; (c) the operant response is controlled by the interplay between facilitatory and suppressive variables that integrate differences between expected (long-term) and experienced (short-term) events; and (d) very-long-term effects are encoded by a consolidated memory that is sensitive to the entire reinforcement history. The model predicts the major qualitative features of operant phenomena and then suggests an experimental test of theoretical predictions about the joint effects of reinforcement probability and amount of training on operant choice. We hypothesize that the set of elementary principles that we propose may help resolve the long-standing debate about the fundamental variables controlling operant conditioning.     

Determinants of contrast in the signal-key procedure: Evidence against additivity theory

Two experiments are reported that challenge the interpretation of previous results with the signal-key procedure, in which the discriminative stimuli are located on a response key different from the key associated with the operant response requirement. Experiment 1 replicated the procedure of Keller (1974), and found that contrast effects on the operant key occurred reliably for only one of four subjects. High rates to the signal key initially occurred for only one subject, but modifications of the procedure produced substantial rates to the signal key for all subjects. In all cases, however, signal-key behavior was greatly reduced by the addition of a changeover delay which prevented reinforcement within 2 seconds of the last peck to the signal key, suggesting that signal-key pecking was maintained primarily by adventitious reinforcement. Experiment 2 modified the signal-key procedure by using three response keys, so that the discriminative stimuli on the signal key controlled different responses during all phases of training. With this modification, reliable contrast effects on the operant key occurred for all subjects, suggesting that the failure to find contrast in previous studies has been due to the confounding of changes in the discrimination requirements with changes in relative rate of reinforcement. The results challenge the additivity theory of contrast, and suggest that “elicited” behavior plays a minor role, if any, in the determination of contrast effects in multiple schedules.     

Effects of delayed reinforcement on infant vocalization rate.

Three previous studies have failed to demonstrate conditioning in infants using a 3-s delay of reinforcement. The effects of a delayed reinforcement schedule on vocalization rates therefore were explored in a single-subject repeated-reversal experimental design for 3 4- to 6-month-old normally developing infants. Each infant received delayed social reinforcement from his or her parent for vocalizing. The comparison condition was a schedule of differential reinforcement of behavior other than vocalizations to control for elicitation by social stimulation. An operant level of infant vocalizations was the initial condition, after which the differential reinforcement schedule was implemented in an across-subjects multiple baseline design. Infants' vocalization rates increased above levels measured during differential reinforcement following onset of the delayed reinforcement condition. Also, vocalization rates decreased during differential reinforcement compared to operant levels. The successful use of delayed reinforcement schedules with infants in this study, as opposed to others, is discussed in terms of procedural differences among them.