APPLIED IMPLICATIONS OF REINFORCEMENT HISTORY EFFECTS

Although the influence of reinforcement history is a theoretical focus of behavior analysis, the specific behavioral effects of reinforcement history have received relatively little attention in applied research and practice. We examined the potential effects of reinforcement history by reviewing nonhuman, human operant, and applied research and interpreted the findings in relation to possible applied significance. The focus is on reinforcement history effects in the context of reinforcement schedules commonly used either to strengthen behavior (e.g., interval schedules) or commonly used to decrease behavior (e.g., extinction).     

Human sensitivity to reinforcement in operant choice: How much do consequences matter?

The results of many human operant conditioning experiments appear to show that humans are less sensitive than nonhumans to operant consequences, suggesting species discontinuities in basic behavioral processes. A reanalysis of 31l data sets from 25 studies employing variable-interval schedules of reinforcement designed to assess sensitivity to reinforcement corroborates the claim that human behavioral allocation among alternatives often deviates from predictions based on rates of experimentally programmed consequences. Close inspection of the studies in question, however, suggests that methodological issues contribute heavily to the differences noted so far between humans and nonhumans and that an explanation based upon species discontinuities is not tenable.     

Computer program to generate operant schedules

A computer program for programming schedules of reinforcement is described. Students can use the program to experience schedules of reinforcement that are typically used with nonhuman subjects. A cumulative recording of a student’s responding can be shown on the screen and/or printed with the computer’s printer. The program can also be used to program operant schedules for animal subjects. The program was tested with human subjects experiencing fixed ratio, variable ratio, fixed interval, and variable interval schedules. Performance for human subjects on a given schedule was similar to performance for nonhuman subjects on the same schedule.     

Stimuli, reinforcers, and behavior: an integration

We propose that a fundamental unit of behavior is the concurrent discriminated operant, and we discuss in detail a quantitative model of the concurrent three-term contingency that is based on the notion that an animal's behavior is controlled to differing extents by both stimulus—behavior and behavior—reinforcer relations. We show how this model can describe performance in a variety of experimental procedures: conditional discrimination and matching to sample, both with and without reinforcement for responses that are traditionally identified as errors; conditional discrimination with more than two stimuli and choice alternatives; delayed matching to sample and delayed reinforcement in matching to sample; second-order and complex conditional discrimination; and multiple and concurrent schedules. Although the model is incomplete in its coverage, and may be incorrect, we believe that this conceptual approach will bear fruit in the development of behavior theory.     

Effects of fixed and variable ratios on human behavioral variability.

The effect that ratio schedules of reinforcement had upon variability of responding was investigated in college students. Subjects were paid $0.02 contingent upon completion of eight presses, distributed in any combination across two push buttons; 256 different sequences were possible. Sequence emission was reinforced according to fixed- and variable-ratio schedules. Ratio requirements of 1, 2, 4 and 8 were presented in alternate components of a multiple schedule. The variability engendered by variable-ratio schedules was also compared to that engendered by fixed ratios. Variability increased with ratio size, irrespective of whether the schedule requirement was fixed or variable. The data demonstrate the similarity between the determinants of human and nonhuman variability, and they illustrate the role of ratio size in determining variability in operant behavior.     

Analyzing the reinforcement process at the human level: Can application and behavioristic interpretation replace laboratory research?

Critics have questioned the value of human operant conditioning experiments in the study of fundamental processes of reinforcement. Contradictory results from human and animal experiments have been attributed to the complex social and verbal history of the human subject. On these grounds, it has been contended that procedures that mimic those conventionally used with animal subjects represent a "poor analytic preparation" for the explication of reinforcement principles. In defending the use of conventional operant methods for human research, we make three points: (a) Historical variables play a critical role in research on processes of reinforcement, regardless of whether the subjects are humans or animals. (b) Techniques are available for detecting, analyzing, and counteracting such historical and extra-experimental influences; these include long-term observations, steady state designs, and, when variables are not amenable to direct control (e.g., age, gender, species), selection of subjects with common characteristics. (c) Other forms of evidence that might be used to validate conditioning principles-applied behavior analysis and behavioristic interpretation-have inherent limitations and cannot substitute for experimental analysis. We conclude that human operant conditioning experiments are essential for the analysis of the reinforcement process at the human level, but caution that their value depends on the extent to which the traditional methods of the experimental analysis of behavior are properly applied.     

Comparing choices and variations in people and rats: Two teaching experiments

Two pairs of experiments enabled students to compare their own operant behaviors with those of rats. The students played computer games for points, and the rats pressed levers for food. The first pair of experiments showed that, under concurrent schedules of reinforcement, relative frequencies of choices between two alternatives increased linearly in rats and people as functions of relative frequencies of reinforcement, with similar biases and undermatching observed in both species. The second pair of experiments showed that behavioral variability was controlled by reinforcers contingent on variability, this again true for both species. These experiments helped demonstrate the relevance of animal operant research to an explanation of human operant behavior.     

Control rate of response or reinforcement and amphetamine's effect on behavior.

The roles of control response rate and reinforcement frequency in producing amphetamine's effect on operant behavior were evaluated independently in rats. Two multiple schedules were arranged in which one variable, either response rate or reinforcement frequency, was held constant and the other variable manipulated. A multiple differential-reinforcement-of-low-rate seven-second yoked variable-interval schedule was used to equate reinforcement frequencies at different control response rates between multiple-schedule components. Amphetamine increased responding under the variable-interval component. In contrast, amphetamine decreased responding equivalently between components of a multiple random-ratio schedule that produced similar control response rates at different reinforcement frequencies. The results provide experimental support to the rate-dependency principle that control rate of responding is an important determinant of amphetamine's effect on operant behavior.     

Human neuropsychology: Some differences between korsakoff and normal operant performance

Summary Operant reinforcement schedules were used to investigate the effects of changes in reinforcement rates on the behavior of alcoholic Korsakoff (amnesic) patients and normal control subjects. In one test, both groups were exposed to pairs of variable-interval (VI) reinforcement schedules which operated concurrently. The distribution of reinforcements controlled the distribution of responses of normal subjects to a significantly greater extent than those of the Korsakoff patients. In a second test, two VI reinforcement schedules were arranged to run one at a time in succession. Reinforcement-rate differences with the successive (multiple) schedules did not produce corresponding response-rate differences for either group of subjects. The findings stress the complexity of the alcoholic Korsakoff syndrome, and emphasize the need to consider possible motivational abnormalities as determinants of alcohol-induced amnesia. In addition, results underscore both the value and the limitations of using operant procedures in human learning research.Supported in part by PHS Grants NS 07615 and NS 06209, by PHS RCDA K04 NS 00161 to M. O. Berman, and by the Medical Research Service of the Veterans Administration. Address offprint requests to M.O. Berman, BVAMC, 150 So. Huntington Ave., Boston, MA. 02130, USA     

Performances on ratio and interval schedules of reinforcement: Data and theory

Two differences between ratio and interval performance are well known: (a) Higher rates occur on ratio schedules, and (b) ratio schedules are unable to maintain responding at low rates of reinforcement (ratio “strain”). A third phenomenon, a downturn in response rate at the highest rates of reinforcement, is well documented for ratio schedules and is predicted for interval schedules. Pigeons were exposed to multiple variable-ratio variable-interval schedules in which the intervals generated in the variable-ratio component were programmed in the variable-interval component, thereby “yoking” or approximately matching reinforcement in the two components. The full range of ratio performances was studied, from strained to continuous reinforcement. In addition to the expected phenomena, a new phenomenon was observed: an upturn in variable-interval response rate in the midrange of rates of reinforcement that brought response rates on the two schedules to equality before the downturn at the highest rates of reinforcement. When the average response rate was corrected by eliminating pausing after reinforcement, the downturn in response rate vanished, leaving a strictly monotonic performance curve. This apparent functional independence of the postreinforcement pause and the qualitative shift in response implied by the upturn in variable-interval response rate suggest that theoretical accounts will require thinking of behavior as partitioned among at least three categories, and probably four: postreinforcement activity, other unprogrammed activity, ratio-typical operant behavior, and interval-typical operant behavior.     

Marc Richelle (1930–2021) and the Study of Temporal Regulation of Behaviour in Animals

This article discusses the contribution of Marc Richelle to the study of temporal regulation of behaviour in animals. Richelle was a pioneer of behavioural pharmacology in Europe in the 1960s, and some of his early pharmacological experiments, particular those involving chlordiazepoxide, are discussed. Richelle frequently tested drug effects on performance on fixed-interval (FI) and differential reinforcement of low rate (DRL) schedules. Much of his later work, conducted with Helga Lejeune, involved cross-species comparisons of performance on FI and DRL, and often focused on potential differences between “timing competence” and “timing performance”. His work provided an unrivalled body of research on operant behaviour in different species, involving research on animals as different as cats and fish. Much of the work was reviewed in Richelle and Lejeune’s 1980 book Time in Animal Behaviour, which contained particularly influential accounts of collateral behaviour and inter-species comparisons.     

The dynamics of operant conditioning

Existing models of operant learning are relatively insensitive to historical properties of behavior and applicable to only limited data sets. This article proposes a minimal set of principles based on short-term and long-term memory mechanisms that can explain the major static and dynamic properties of operant behavior in both single-choice and multiresponse situations. The critical features of the theory are as follows: (a) The key property of conditioning is assessment of the degree of association between responses and reinforcement and between stimuli and reinforcement; (b) the contingent reinforcement is represented by learning expectancy, which is the combined prediction of response-reinforcement and stimulus-reinforcement associations; (c) the operant response is controlled by the interplay between facilitatory and suppressive variables that integrate differences between expected (long-term) and experienced (short-term) events; and (d) very-long-term effects are encoded by a consolidated memory that is sensitive to the entire reinforcement history. The model predicts the major qualitative features of operant phenomena and then suggests an experimental test of theoretical predictions about the joint effects of reinforcement probability and amount of training on operant choice. We hypothesize that the set of elementary principles that we propose may help resolve the long-standing debate about the fundamental variables controlling operant conditioning.     

Operant and nonoperant vocal responding in the mynah: Complex schedule control and deprivation-induced responding

Several recent studies have been concerned with operant responses that are also affected by nonoperant factors, (e.g., biological constraints, innate behavior patterns, respondent processes). The major reason for studying mynah vocal responding concerned the special relation of avian vocalizations to nonoperant emotional and reflexive systems. The research strategy was to evaluate operant and nonoperant control by comparing the schedule control obtained with the vocal response to that characteristic of the motor responses of other animals. We selected single, multiple, and chain schedules that ordinarily produce disparate response rates at predictable times. In multiple schedules with one component where vocal responding (“Awk”) was reinforced with food (fixed-ratio or fixed-interval schedule) and one where the absence of vocal responding was reinforced (differential reinforcement of other behavior), response rates never exceeded 15 responses per minute, but clear schedule differences developed in response rate and pause time. Nonoperant vocal responding was evident when responding endured across 50 extinction sessions at 25% to 40% of the rate during reinforcement. The “enduring extinction responding” was largely deprivation induced, because the operant-level of naive mynahs under food deprivation was comparable in magnitude, but without deprivation the operant level was much lower. Food deprivation can induce vocal responding, but the relatively precise schedule control indicated that operant contingencies predominate when they are introduced.     

The role of verbal behavior in human learning: II. Developmental differences

When children in four different age ranges operated a response device, reinforcers were presented according to fixed-interval schedules ranging in value from 10 to 70 seconds. Only the behavior of the subjects in the youngest of the four groups, the preverbal infants, resembled that of other animal species. The children in age ranges 5 to 6½ and 7½ to 9 years exhibited either the low-rate or high-rate response patterns typical of human adults. Those who showed the low-rate pattern reported a time-based formulation of the contingencies and some of them were observed to occasionally count out the interval before responding. The performance of children aged 2½ to 4 years differed from that of both infants and older children, though containing some patterning elements similar to those produced by the older and younger subjects. The predominant response pattern of the infants consisted of a pause after reinforcement followed by an accelerated rate of responding that terminated when the next reinforcer was delivered. Analysis of postreinforcement-pause duration and response rate showed that infant performance, but not that of the older children, consistently exhibited the same kinds of schedule sensitivity observed in animal behavior. The evidence supports the suggestion that the development of verbal behavior greatly alters human operant performance and may account for many of the differences found between human and animal learning.     

A half century of scalloping in the work habits of the United States Congress

It has been suggested that the work environment of the United States Congress bears similarity to a fixed-interval reinforcement schedule. Consistent with this notion, Weisberg and Waldrop (1972) described a positively accelerating pattern in annual congressional bill production (selected years from 1947 to 1968) that is reminiscent of the scalloped response pattern often attributed to fixed-interval schedules, but their analysis is now dated and does not bear on the functional relations that might yield scalloping. The present study described annual congressional bill production over a period of 52 years and empirically evaluated predictions derived from four hypotheses about the mechanisms that underlie scalloping. Scalloping occurred reliably in every year. The data supported several predictions about congressional productivity based on fixed-interval schedule performance, but did not consistently support any of three alternative accounts. These findings argue for the external validity of schedule-controlled operant behavior as measured in the laboratory. The present analysis also illustrates a largely overlooked role for applied behavior analysis: that of shedding light on the functional properties of behavior in uncontrolled settings of considerable interest to the public.     

Within-session changes in operant responding when gerbils (Meriones unguiculatus) serve as subjects

Recent research has demonstrated that rate of responding frequently changes in a robust and systematic manner during experimental sessions in which organisms engage in operant responding. One potential cause for these changes in response rate is that levels of exploration change during experimental sessions and that high levels of exploration interfere with operant responding. Several studies have shown that gerbils (Meriones unguiculatus) explore at a constant rate during experimental sessions. The present study examined the response pattern produced by gerbils responding for food delivered by several simple schedules of reinforcement. Results indicated that robust changes (between 200 and 400%) in response rate occurred during the experimental sessions. These data argue against a role for exploration in the production of within-session changes in operant response rate.     

Effects of delayed reinforcement on infant vocalization rate.

Three previous studies have failed to demonstrate conditioning in infants using a 3-s delay of reinforcement. The effects of a delayed reinforcement schedule on vocalization rates therefore were explored in a single-subject repeated-reversal experimental design for 3 4- to 6-month-old normally developing infants. Each infant received delayed social reinforcement from his or her parent for vocalizing. The comparison condition was a schedule of differential reinforcement of behavior other than vocalizations to control for elicitation by social stimulation. An operant level of infant vocalizations was the initial condition, after which the differential reinforcement schedule was implemented in an across-subjects multiple baseline design. Infants' vocalization rates increased above levels measured during differential reinforcement following onset of the delayed reinforcement condition. Also, vocalization rates decreased during differential reinforcement compared to operant levels. The successful use of delayed reinforcement schedules with infants in this study, as opposed to others, is discussed in terms of procedural differences among them.     

Effects of instructions and reinforcement-feedback on human operant behavior maintained by fixed-interval reinforcement

In three experiments, human subjects were trained on a five-component multiple schedule with different fixed intervals of monetary reinforcement scheduled in the different components. Subjects uninstructed about the fixed-interval schedules manifested high and generally equivalent rates regardless of the particular component. By comparison, subjects given instructions about the schedules showed orderly progressions of rates and temporal patterning as a function of the interreinforcement intervals, particularly when feedback about reinforcement was delivered but also when reinforcement-feedback was withheld. Administration of the instructions-reinforcement combination to subjects who had already developed poorly differentiated behavior, however, did not make their behavior substantially better differentiated. When cost was imposed for responding, both instructed and uninstructed subjects showed low and differentiated rates regardless of their prior histories. It was concluded that instructions can have major influences on the establishment and maintenance of human operant behavior.     

Nonstable concurrent choice in pigeons

Six pigeons were trained on concurrent variable-interval schedules in which the arranged reinforcer ratios changed from session to session according to a 31-step pseudorandom binary sequence. This procedure allows a quantitative analysis of the degree to which performance in an experimental session is affected by conditions in previous sessions. Two experiments were carried out. In each, the size of the reinforcer ratios arranged between the two concurrent schedules was varied between 31-step conditions. In Experiment 1, the concurrent schedules were arranged independently, and in Experiment 2 they were arranged nonindependently. An extended form of the generalized matching law described the relative contribution of past and present events to present-session behavior. Total performance in sessions was mostly determined by the reinforcer ratio in that session and partially by reinforcers that had been obtained in previous sessions. However, the initial exposure to the random sequence produced a lower sensitivity to current-session reinforcers but no difference in overall sensitivity to reinforcement. There was no evidence that the size of the reinforcer ratios available on the concurrent schedules affected either overall sensitivity to reinforcement or the sensitivity to reinforcement in the current session. There was also no evidence of any different performance between independent and nonindependent scheduling. Because of these invariances, this experiment validates the use of the pseudorandom sequence for the fast determination of sensitivity to reinforcement.     

Handwriting as an operant

An apparatus was designed to monitor handwriting behavior. Two subjects were studied under various schedules of monetary reinforcement for handwriting. The different schedules engendered and maintained distinctive response patterns but the rates of sustained responding did not vary across schedules. The development of fixed-interval performance following continuous reinforcement resembled the same transition in lower animals. In one subject, availability of reading material interacted with the schedule to determine response pattern. It was suggested that handwriting may be a more appropriate response for the experimental analysis of human behavior than the more frequently used button-pushing or lever-pulling responses.