Within-session rates of responding when reinforcer magnitude is changed within the session

In the present experiment, the authors investigated the idea that within-session changes in operant response rates occur because subjects sensitize and then habituate to the reinforcer. If that is true, then altering an aspect of the reinforcer within the session should alter the observed within-session responding. The authors tested that idea by having rats press a lever for 2 food-pellet reinforcers delivered by a variable-interval 120-s schedule during 60-min baseline sessions. In treatment conditions, the magnitude of the reinforcer was halved (1 pellet) or doubled (4 pellets) 10, 20, 30, 40, or 50 min into the session. That magnitude of reinforcement then remained in effect for the rest of the session. Altering reinforcer magnitude altered the rates of responding within the session in a fashion consistent with the habituation explanation, that is, response rates increased, relative to baseline, when the magnitude of reinforcement was increased. They decreased when the magnitude was decreased. Those results were seemingly inconsistent with the competing idea that within-session decreases in responding rates are produced by satiation.     

Within-session Response Patterns On Conjoint Variable-interval Variable-time Schedules

Operant responding often changes within sessions, even when factors such as rate of reinforcement remain constant. The present study was designed to determine whether within-session response patterns are determined by the total number of reinforcers delivered during the session or only by the reinforcers earned by the operant response. Four rats pressed a lever and 3 pigeons pecked a key for food reinforcers delivered by a conjoint variable-interval variable-time schedule. The overall rate of reinforcement of the conjoint schedule varied across conditions from 15 to 480 reinforcers per hour. When fewer than 120 reinforcers were delivered per hour, the within-session patterns of responding on conjoint schedules were similar to those previously observed when subjects received the same total number of reinforcers by responding on simple variable-interval schedules. Response patterns were less similar to those observed on simple variable-interval schedules when the overall rate of reinforcement exceeded 120 reinforcers per hour. These results suggest that response-independent reinforcers can affect the within-session pattern of responding on a response-dependent schedule. The results are incompatible with a response-based explanation of within-session changes in responding (e.g., fatigue), but are consistent with both reinforcer-based (e.g., satiation) and stimulus-based (e.g., habituation) explanations.     

Response elimination: A comparison of four procedures

Four procedures for eliminating an operant response were compared within a multiple schedule. Response reduction was most rapid when reinforcement was provided for a specific alternative response. The decline in responding produced by extinction and differential reinforcement of other behavior was similar. Responding initially increased and then gradually decreased when reinforcers were delivered independently of responding at fixed times. Removal of reinforcement from the alternative-response and DRO procedures did not result in recovery of the target response. However, the shift from response-independent dilivery of reinforcers to extinction caused an initial increase in responding.     

Operant and nonoperant vocal responding in the mynah: Complex schedule control and deprivation-induced responding

Several recent studies have been concerned with operant responses that are also affected by nonoperant factors, (e.g., biological constraints, innate behavior patterns, respondent processes). The major reason for studying mynah vocal responding concerned the special relation of avian vocalizations to nonoperant emotional and reflexive systems. The research strategy was to evaluate operant and nonoperant control by comparing the schedule control obtained with the vocal response to that characteristic of the motor responses of other animals. We selected single, multiple, and chain schedules that ordinarily produce disparate response rates at predictable times. In multiple schedules with one component where vocal responding (“Awk”) was reinforced with food (fixed-ratio or fixed-interval schedule) and one where the absence of vocal responding was reinforced (differential reinforcement of other behavior), response rates never exceeded 15 responses per minute, but clear schedule differences developed in response rate and pause time. Nonoperant vocal responding was evident when responding endured across 50 extinction sessions at 25% to 40% of the rate during reinforcement. The “enduring extinction responding” was largely deprivation induced, because the operant-level of naive mynahs under food deprivation was comparable in magnitude, but without deprivation the operant level was much lower. Food deprivation can induce vocal responding, but the relatively precise schedule control indicated that operant contingencies predominate when they are introduced.     

Control rate of response or reinforcement and amphetamine's effect on behavior.

The roles of control response rate and reinforcement frequency in producing amphetamine's effect on operant behavior were evaluated independently in rats. Two multiple schedules were arranged in which one variable, either response rate or reinforcement frequency, was held constant and the other variable manipulated. A multiple differential-reinforcement-of-low-rate seven-second yoked variable-interval schedule was used to equate reinforcement frequencies at different control response rates between multiple-schedule components. Amphetamine increased responding under the variable-interval component. In contrast, amphetamine decreased responding equivalently between components of a multiple random-ratio schedule that produced similar control response rates at different reinforcement frequencies. The results provide experimental support to the rate-dependency principle that control rate of responding is an important determinant of amphetamine's effect on operant behavior.     

Response-initiated imaging of operant behavior using a digital camera

A miniature digital camera, QuickCam Pro 3000, intended for use with video e-mail, was modified so that snapshots were triggered by operant behavior emitted in a standard experimental chamber. With only minor modification, the manual shutter button on the camera was replaced with a simple switch closure via an I/O interface controlled by a PC computer. When the operant behavior activated the I/O switch, the camera took a snapshot of the subject's behavior at that moment. To illustrate the use of the camera, a simple experiment was designed to examine stereotypy and variability in topography of operant behavior under continuous reinforcement and extinction in 6 rats using food pellets as reinforcement. When a rat operated an omnidirectional pole suspended from the ceiling, it also took a picture of the topography of its own behavior at that moment. In a single session after shaping of pole movement (if necessary), blocks of continuous reinforcement, in which each response was reinforced, alternated with blocks of extinction (no reinforcement), with each block ending when 20 responses had occurred. The software supplied with the camera automatically stored each image and named image files successively within a session. The software that controlled the experiment also stored quantitative data regarding the operant behavior such as consecutive order, temporal location within the session, and response duration. This paper describes how the two data types--image information and numerical performance characteristics-can be combined for visual analysis. The experiment illustrates in images how response topography changes during shaping of pole movement, how response topography quickly becomes highly stereotyped during continuous reinforcement, and how response variability increases during extinction. The method of storing digital response-initiated snapshots should be useful for a variety of experimental situations that are intended to examine behavior change and topography.     

The effect of reinforcement differences on choice and response distribution during stimulus compounding

In Experiments I and II, rats were trained to respond on one lever during light and another during tone. The absence of tone and light controlled response cessation. In the multiple schedule of Experiment I, all reinforcements were received for responding in tone or light; in the chain schedule of Experiment II, all reinforcements were received in no tone + no light for not responding. Experiment I subjects, for which tone and light were associated with response and reinforcement increase, responded significantly more to tone-plus-light than to tone or light alone (additive summation). Experiment II subjects, for which tone and light were associated with response increase and reinforcement decrease, responded comparably to tone, light, and tone + light. Thus, additive summation was observed when stimulus-response and stimulus-reinforcer associations in tone and light were both positive, but not when they were conflicting. All subjects in both experiments responded predominantly on the light-correlated lever during tone + light, even when light intensity was reduced in testing. Furthermore, when a light was presented to a subject engaged in tone-associated responding, all subjects immediately switched the locus of responding to the light-correlated lever. No change in locus occurred when a tone was presented to a subject engaged in light-associated responding, irrespective of the stimulus-reinforcer association conditioned to tone. The light-lever preference in tone + light indicates that the heightened responding observed in Experiment I was not the summation of tone-associated behavior with light-associated behavior. Rather, it appears to be the result of a facilitation of one operant (light-associated responding) by the reinforcement-associated cue for the other.     

Effects of differential rates of alternative reinforcement on resurgence of human behavior

Despite the success of exposure‐based psychotherapies in anxiety treatment, relapse remains problematic. Resurgence, the return of previously eliminated behavior following the elimination of an alternative source of reinforcement, is a promising model of operant relapse. Nonhuman resurgence research has shown that higher rates of alternative reinforcement result in faster, more comprehensive suppression of target behavior, but also in greater resurgence when alternative reinforcement is eliminated. This study investigated rich and lean rates of alternative reinforcement on response suppression and resurgence in typically developing humans. In Phase 1, three groups (Rich, n = 18; Lean, n = 18; Control, n = 10) acquired the target response. In Phase 2, target responding was extinguished and alternative reinforcement delivered on RI 1 s, RI 3 s, and extinction schedules, respectively. Resurgence was assessed during Phase 3 under extinction conditions for all groups. Target responding was suppressed most thoroughly in Rich and partially in Lean. Target responding resurged in the Rich and Lean groups, but not in the Control group. Between groups, resurgence was more pronounced in the Rich group than the Lean and Control groups. Clinical implications of these findings, including care on the part of clinicians when identifying alternative sources of reinforcement, are discussed.     

Positive and negative conditioned suppression in the pigeon: Effects of the locus and modality of the CS

In Experiment I, four pigeons were exposed to trials in which a 12-sec key light illumination was followed by free food. These trials were superimposed upon a baseline of key pecking for food reinforcement on a variable-interval schedule. When the signal for food was on the operant key, response rate was substantially higher during the signal than during the baseline procedure. When the signal was on a second, signal key, operant responding was suppressed during the signal and substantial pecking of the signal key occurred. The sum of signal key and operant key pecks far exceeded the operant baseline rate of responding. An explanation of opposite results obtained with rats and pigeons as subjects in experiments of this type was suggested in terms of the spatial relation between the signal for free food and the operant target which usually characterizes these experiments. Experiment II assessed the importance of signal location when shock rather than food was the US. Suppression of operant key pecking was unaffected by signal location. Experiment III assessed the relative effectiveness of visual and auditory stimuli (clicks) as signals for food and shock, and found that all combinations of signal and US were equally effective in suppressing operant key responding. The three experiments together suggested that the identification of important effects of species—typical behavior in one experimental situation does not imply that there will be like effects in similar situations.     

Effects of concurrent response-independent reinforcement on fixed-interval schedule performance

In three experiments, behavior maintained by fixed-interval schedules changed when response-independent reinforcement was delivered concurrently according to fixed- or variable-time schedules. In Experiment I, a pattern of positively accelerated responding during fixed interval was changed to a linear pattern when response-independent reinforcement occurred under a variable-time schedule. Overall response rates (total responses/total time) decreased as the frequency of response-independent reinforcement increased. Experiment II showed that the response-rate changes in the first experiment were controlled by the response-reinforcer relation, but the changes in patterns of responding were similar whether concurrently available reinforcement at varying times was response-dependent or response-independent. In the final experiment, the addition of response-independent reinforcement at fixed times to a fixed-interval schedule resulted in changes in both local and overall response rates and in the occurrence of positively accelerated responding between reinforcements. These results suggest that the temporal distribution of reinforcers determines response patterns and that both the response-reinforcement dependency and the schedule of reinforcement determine overall response rates during concurrently scheduled response-dependent and response-independent reinforcement.     

Effects of reinforcer consumption and magnitude on response rates during noncontingent reinforcement

Results of previous research on the effects of noncontingent reinforcement (NCR) have been inconsistent when magnitude of reinforcement was manipulated. We attempted to clarify the influence of NCR magnitude by including additional controls. In Study 1, we examined the effects of reinforcer consumption time by comparing the same magnitude of NCR when session time was and was not corrected to account for reinforcer consumption. Lower response rates were observed when session time was not corrected, indicating that reinforcer consumption can suppress response rates. In Study 2, we first selected varying reinforcer magnitudes (small, medium, and large) on the basis of corrected response rates observed during a contingent reinforcement condition and then compared the effects of these magnitudes during NCR. One participant exhibited lower response rates when large-magnitude reinforcers were delivered; the other ceased responding altogether even when small-magnitude reinforcers were delivered. We also compared the effects of the same NCR magnitude (medium) during 10-min and 30-min sessions. Lower response rates were observed during 30-min sessions, indicating that the number of reinforcers consumed across a session can have the same effect as the number consumed per reinforcer delivery. These findings indicate that, even when response rate is corrected to account for reinforcer consumption, larger magnitudes of NCR (defined on either a per-delivery or per-session basis) result in lower response rates than do smaller magnitudes.     

Sexual reinforcement in the female rat.

Sexual reinforcement in the female rat was studied in a preparation that allowed continuous operant responding for access to a male rat leading to intromission. Experiment 1 used a high operant level nose-poke response to test the possible reinforcing effects of some components of access to a male. A simple tone stimulus used as a conditioned reinforcer and two odor stimuli, target male bedding and emulsified preputial gland, were tested. None of these contingent events altered responding above or below operant level. Access to the male, which was always accompanied by intromission, immediately increased response rate when it was made contingent upon the nose-poke response. Performance on fixed-ratio schedules was erratic, and response rate was low in comparison to typical food-reinforced responding. An interresponse-time analysis indicated, however, that some effect of the ratio contingency may have been present. In Experiment 2, several modifications of the procedure were tested with the objective of creating a more tractable preparation for behavior analysis. Response type and the hormone delivery method were changed, and 2 target males were used instead of 1. The latter tripled the average number of reinforcers earned in a single session. Differences between sexual and other reinforcers are discussed in terms of procedural, quantitative, and motivational aspects of the sexual reinforcement procedure.     

Effects of response pacing on conditioned suppression

In this experiment, which employed a balanced design with two rat subjects, the frequency of reinforcement remained constant while the rate of operant responding was varied by means of a response pacing technique. At each of three response rates, 1-min. periods of noise were presented, and, as these periods ended, a slight unavoidable shock was delivered to the rat. This procedure resulted in suppression of the operant responding during the periods of noise. This behavioural change was measured by a suppression ratio, essentially a comparison of the response rates in the presence and absence of the noise. The suppression ratios varied in a systematic way during the experiment, denoting most conditioned suppression when the baseline rate of responding was high, and least suppression when this was low. It is therefore concluded that response rate in one factor determining the degree of conditioned suppression in this controlled experiment. The conclusion is corroborated by absolute measures of responding during the pre-shock periods of noise.     

The influence of upcoming food-pellet delivery on subjects' responding for 1% sucrose reinforcement delivered by concurrent random-interval schedules

Researchers have demonstrated that rats' rates of operant responding that are maintained by 1% liquid-sucrose reinforcement will increase if food-pellet reinforcement is upcoming within the same session. The authors investigated whether a similar induction effect would be observed when rats pressed a lever for 1% sucrose that was delivered by concurrent random-interval schedules of reinforcement. Results demonstrated that upcoming noncontingent food-pellet delivery increased absolute response rates on the concurrent schedules in 10 of 12 possible instances. Upcoming food-pellet delivery also increased subjects' sensitivity to reinforcement on the concurrent schedules, as measured by the generalized matching law (W. M. Baum, 1974), in 5 of 6 possible instances. The present results extended the finding of induction to responding on concurrent schedules. They also added to evidence suggesting that the effect occurs because the reinforcing value of the weak reinforcer (i.e., the 1% sucrose) has been increased.     

Economic and biological influences on key pecking and treadle pressing in pigeons

Pigeons were studied on a two-component multiple schedule in which the required operant was, in different conditions, biologically relevant (i.e., key pecking) or nonbiologically relevant (i.e., treadle pressing). Responding was reinforced on a variable-interval (VI) 2-min schedule in both components. In separate phases, additional food was delivered on a variable-time (VT) 15-s schedule (response independent) or a VI 15-s schedule (response dependent) in one of the components. The addition of response-independent food had different effects on responding depending on the operant response and on the frequency with which the components alternated. When components alternated frequently (every 10 s), all pigeons keypecked at a much higher rate during the component with the additional food deliveries, whether response dependent or independent. In comparison, treadle pressing was elevated only when the additional food was response dependent; rate of treadling was lower when the additional food was response independent. When components alternated infrequently (every 20 min), pigeons key pecked at high rates at points of transition into the component with the additional food deliveries. Rate of key pecking decreased with time spent in the 20-min component when the additional food was response independent, whereas rate of pecking remained elevated in that component when the additional food was response dependent. Under otherwise identical test conditions, rate of treadle pressing varied only as a function of its relative rate of response-dependent reinforcement. Delivery of response-independent food thus had different, but predictable, effects on responding depending on which operant was being studied, suggesting that animal-learning procedures can be integrated with biological considerations without the need to propose constraints that limit general laws of learning.     

Predator videos and electric shock function as punishers for zebrafish (Danio rerio)

Zebrafish (Danio rerio) are a promising animal model for studying the effects of gene–environment interactions on behavior. Two experiments were conducted to assess punishment effects of presenting predator videos (Indian leaf fish; Nandus nandus) and electric shock on operant approach responses in zebrafish. In Experiment 1, the predator video and shock stimuli were presented upon a response maintained by a single variable‐interval schedule of food reinforcement in different groups of fish. In Experiment 2, the predator video and shock stimuli were presented upon one of two response alternatives maintain by concurrently available variable‐interval schedules of food reinforcement in different groups of fish. Responding decreased when the predator video and shock stimuli were presented relative to their absence in both experiments. Moreover, responding on an unpunished alternative did not reliably decrease in Experiment 2. These results indicate that the decrease in responding resulted from the punishment contingency rather than from elicited species‐specific defense responses or conditioned avoidance. Thus, the predator video and electric shock functioned as punishers of operant behavior for zebrafish. Identifying punishers for this species could lead to research on how gene–environment interactions influence individual differences in sensitivity to punishment.     

The effect of a teacher's presence on the classroom behavior of conduct-problem children

Six institutionalized conduct-problem children performed in a classroom under three reinforcement conditions: (1) noncontingent reinforcement: (2) reinforcement for being on task and (3) reinforcement for the accuracy and rate fo their academic behavior. Within each of these conditions, the teacher was either present throughout the class session or absent for a portion of the session. In the teacher's absence, on-task behavior was markedly reduced and disruption was markedly increased, regardless of the reinforcement condition in operation. In contrast, the teacher's absence had no effect on academic accuracy and had a major effect on academic rate only when reinforcement was delivered noncontingently. Furthermore, the extent to which the children became disruptive in the teacher's absence was reduced when reinforcement was contingent upon academic accuracy and rate, instead of being contingent upon being on task or delivered noncontingently. It is suggested that the reinforcement of academic behavior, rather than on-task behavior or classroom social behavior, not only will improve the latter behaviors as well, but possibly also make them less dependent upon the presence and continued surveillance of the teacher.     

Conditioned suppression or facilitation as a function of the behavioral baseline

Rats were exposed to a multiple schedule of reinforcement. During one component, a bar-press was followed by reinforcement only if it occurred between 15 and 20 sec after the previous response. This differential-reinforcement-of-low-rate (DRL) schedule produced a typical slow rate of responding. During the other component, reinforcement followed the first response to be emitted during limited periods of time which occurred at fixed intervals. These fixed-interval schedules with a limited hold produced higher response rates, described as `interval' or `ratio-like' behavior. Responding during the DRL component increased in frequency during a tone which ended with an unavoidable shock of low intensity, but decreased during the tone when the shock intensity was raised. The `interval' and `ratio-like' responding decreased in frequency during the tone at all shock intensities. Initial acceleration of the DRL responding appeared to be due to adventitious punishment of collateral behavior which was observed between the bar-presses. The more severe conditioned suppression during the fixed-interval components might be the result of the lower probability of reinforcement after any single response.     

Effect of signaled reinforcement on response variability

Three experiments examined the effect of signaling reinforcement on rats' lever pressing on contingencies that reinforced variable responding to extend the exploration of signaled reinforcement to a schedule that has previously not been examined in this respect. In Experiment 1, rats responding on a lag-8 variability schedule with signaled reinforcement displayed greater levels of variability (U values) than rats on the same schedule lacking a reinforcement signal. In Experiment 2, rats responding on a differential reinforcement of least frequent responses schedule also displayed greater operant variability with a signal for reinforcement compared with rats without a reinforcement signal. In Experiment 3, a reinforcement signal decreased the variability of a response sequence when there was no variability requirement. These results offer empirical corroboration that operant variability responds to manipulations in the same manner as do other forms of operant response and that a reinforcement signal facilitates the emission of the required operant.     

Within-session changes in operant responding when gerbils (Meriones unguiculatus) serve as subjects

Recent research has demonstrated that rate of responding frequently changes in a robust and systematic manner during experimental sessions in which organisms engage in operant responding. One potential cause for these changes in response rate is that levels of exploration change during experimental sessions and that high levels of exploration interfere with operant responding. Several studies have shown that gerbils (Meriones unguiculatus) explore at a constant rate during experimental sessions. The present study examined the response pattern produced by gerbils responding for food delivered by several simple schedules of reinforcement. Results indicated that robust changes (between 200 and 400%) in response rate occurred during the experimental sessions. These data argue against a role for exploration in the production of within-session changes in operant response rate.