Independence of response force and reinforcement rate on concurrent variable-interval schedule performance

Five pigeons were trained over 43 experimental conditions on a variety of concurrent variable-interval schedules on which the forces required on the response keys were varied. The results were well described by the generalized matching law with log reinforcement ratios and log force ratios exerting independent (noninteractive) effects on preference. A further analysis using the Akaike criterion, an information-theoretic measure of the efficiency of a model, showed that overall reinforcement rate and overall force requirement did not affect preference. Unlike reinforcement rate changes, force requirement increases did not change the response rate on the alternate key, and an extension of Herrnstein's absolute response rate function for force variation on a single variable-interval schedule is suggested.     

Response Form, Force, And Number: Effects On Concurrent-schedule Performance

Six hens responded on concurrent variable-interval (key-peck) variable-interval (door-push) schedules of reinforcement in which the second-order (fixed-ratio) requirements on the alternatives (Experiment 1) or the required door forces (Experiment 2) were varied. The key-peck and door-push response (measured as fixed-ratio completion) and time data were well described by the generalized matching law. However, the manipulations of fixed-ratio requirement and required response force differed in their effects. The manipulations of fixed-ratio size affected the response and time measures differently, producing fairly constant, multiplicative biases only in terms of response allocation. It was argued that variations in fixed-ratio size necessarily change the time allocated to that response unit, and thus changes in time bias were not necessarily a fundamental effect of changing the ratio. In contrast, the changes in response bias were a fundamental result of changes in ratio size. The response-force manipulations produced similar bias shifts in terms of response and time allocation, but they appeared to combine with relative reinforcement rate to affect choice interactively. Specifically, behavior appeared to be biased towards the least effortful (i.e., key-peck) response, but the increases in door force had a larger effect on bias when the hens were making this response infrequently (on a lean schedule). The different effects of the fixed-ratio and response-force manipulations on concurrent performance were partially accounted for by the differing times required to complete each response unit under those manipulations, but this would not account for the interaction. The interaction would be consonant with increased response effort decreasing the effective value of the associated reinforcement schedule.     

Effect of drugs on response-duration differentiation VII: response-force requirements

Rats were trained to press a lever for at least 1 s but for less than 1.3 s. The force required to press the lever was then increased or decreased by 10, 15, or 20 g. Increases in the force requirements for lever pressing decreased timing accuracy, but decreases in the force requirement had the opposite effect. Accuracy decreases at increasing force requirements were characterized by an increase in the relative frequency of responses that were too short to meet the reinforcement criterion. In contrast, increases in accuracy when the force requirements were decreased were characterized by increases in response durations that met the reinforcement criterion and decreases in the relative frequency of responses that were too short to produce the reinforcer. Phencyclidine (PCP) and methamphetamine produced dose-dependent decreases in accuracy that were associated primarily with increases in the relative frequency of short response durations, although methamphetamine also produced increases in long response durations at some doses. When the effects of PCP were determined with the force requirement increased by 10 g or decreased by 15 g, the cumulative response-duration distribution shifted toward even shorter response durations. When the effects of methamphetamine were determined with the force requirement on the lever increased by 10 g, the cumulative frequency distribution was shifted toward shorter response durations to about the same extent as it had been before force requirements increased; however, when the force required to press the lever was decreased by 15 g, these shifts toward shorter response durations almost completely disappeared. These results show that increases and decreases in the force requirements for lever pressing have different effects on the accuracy of temporal response differentiation.     

Aversive functions of response effort: Fact or artifact?

Historically, effort has been viewed as aversive. Most supporting evidence comes from studies demonstrating increased force/effort requirements reduce operant responding. Changes in force/effort requirements, however, are often accompanied by changes in response definition when mechanical devices are used to define the response. As a consequence, responses measured at one point in a study may go unmeasured at other points. In an alternative approach, we used a continuous measurement strategy that provided a means to fix the threshold force defining the response class and simultaneously allowed independent manipulation of the force criteria required to produce reinforcement. Rats pressed a force transducer according to a fixed‐ratio 5 schedule of food delivery. The criterion force was systematically increased and decreased; the threshold for response detection was constant. When response rates included only criterion responses, overall rate decreased when force requirements increased. By contrast, when all responses, both those meeting force criteria and those that did not (above the threshold but below the criteria for reinforcement) were included in the rate calculation, increases in force increased response rate. Increases in force criteria also increased the maximum force (g) and time‐integral of force (g‐s) of operant behavior. Control conditions showed increases in responding could be explained by the emergence of subcriterion responses, irrespective of force. We conclude that prior results showing effort decreases response rates are due to an artifact arising from inadvertent changes in response definitions. Increases in effort may better be understood as changes in the response:reinforcer payoff owing to the emergence of a subcriterion response class.     

Distinguishing between haloperidol's and decamethonium's disruptive effects on operant behavior in rats: use of measurements that complement response rate.

The behavioral effects of haloperidol (0.04 to 0.16 mg/kg) and nonparalytic doses of decamethonium (0.2 to 0.8 mg/kg) were studied with operant methods that permitted the measurement of response rate, peak force of response, duration of response, and duration of the rat's head entry into the reinforcement dipper well. Type of operant response topography (forelimb press or forelimb grasp-and-pull) and peak force (low or high) required for reinforcement delivery were independent variables. The low-force, press-topography condition yielded qualitatively different profiles for the two drugs. Haloperidol increased peak force and duration of operant response, increased maximum head entry duration, and temporally dissociated forelimb and head entry behavior. Decamethonium decreased force and duration of operant response, did not appreciably affect maximum head entry duration, and did not influence the normal temporal coupling of forelimb and head entry responses. The haloperidol effects were seen as reflections of pseudo-Parkinsonism, not muscle weakness, which appeared to be the primary source of decamethonium's behavioral effects.     

Bouts of responding: the relation between bout rate and the rate of variable-interval reinforcement

By nose poking a lighted key, rats obtained food pellets on either a variable-interval schedule of reinforcement or a schedule that required an average of four additional responses after the end of tile variable-interval component (a tandem variable-interval variable-ratio 4 schedule). With both schedule types, the mean variable interval was varied between blocks of sessions from 16 min to 0.25 min. Total rate of key poking increased similarly as a function of the reinforcer rate for the two schedule types, but response rate was higher with than without the four-response requirement. Analysis of log survivor plots of interresponse times showed that key poking occurred in bouts. The rate of initiating bouts increased as a function of reinforcer rate but was either unaffected or was decreased by adding the four-response requirement. Within-bout response rate was insensitive to reinforcer rate and only inconsistently affected by the four-response requirement. For both kinds of schedule, the ratio of bout time to between-bout pause time was approximately a power function of reinforcer rate, with exponents above and below 1.0.     

A ROLE FOR NEGATIVE REINFORCEMENT OF RESPONSE OMISSION IN PUNISHMENT?

This experiment attempted to disentangle response-rate reductions controlled by the direct suppressive effects of a punisher from those due to negative reinforcement of response omission. Key-peck responding of pigeons was maintained by a conjoint variable-interval 3-min schedule of food presentation variable-interval 30-s schedule of response-dependent electric shock presentation. Omission of responses for 5, 10, or 30 s resulted in the possibility of canceling a scheduled shock. Response rates were a function of required pause duration, with lower rates occurring when longer periods of response omission were required for shock cancellation. These results show that, with several parameters of punishment held constant, response rates were controlled by the negative reinforcement contingency. Such a finding argues for renewed consideration of the role of negative reinforcement in punishment contingencies.     

Effects of differing response-force requirements on food-maintained responding in CD-1 mice

The effect of force requirements on response effort was examined using outbred (CD-1) mice trained to press a disk with their snout. Lateral peak forces greater than 2 g were defined as threshold responses (i.e., all measured responses). Different force requirements were used to define criterion responses (a subclass of threshold responses) that exceeded the requirement. The reinforcer was sweetened, condensed milk, and it was delivered upon response termination. All mice were exposed to two ascending series of criterion force requirements (2, 4, 8, 16, and 32 g). Increasing the force requirement decreased criterion response rates, but increased threshold response rates. The time-integral of force (area under the force-time curve for individual responses, which is proportional to energy expenditure for each response) increased with the increase in the force requirement. These results conflict with the hypothesis that higher force requirements have aversive qualities and suggest that increased force requirements are more analogous to intermittent schedules of reinforcement. These data suggest that estimations of effort or energy expenditure should be measured independently of the force requirement. Individual differences in responding were found for the CD-1 outbred stock.     

Early-extinction effects on a force response of humans

Humans were used to investigate changes in response force occurring soon after reinforcement was eliminated. In Experiment 1, in a 300-s baseline phase, 10 participants received a point for holding down a pressure sensor set to operate at a force equal to 85% of the maximum force the participants exerted during a pretest. Following this, during a 600-s extinction phase, criterion responses had no consequence. In Experiment 2, 6 participants worked on the same task, but (a) points were exchangeable for money and (b) after extinction, the reinforcement baseline phase was reinstated. In Experiment 3, 6 participants completed the same task as in Experiment 2, but the required minimum force was 60% of the maximum force exerted during the pretest. In each experiment, increases in response force relative to the mean and peak force exerted during the last 100 s of baseline were observed in most participants when force responses were aggregated into short sample intervals, but less so with longer ones. The increases, however, were not systematic across or within participants, questioning the generality of and the criteria for demonstrating an extinction burst.     

Local response-rate constancy on concurrent variable-interval schedules of reinforcement

Concurrent variable-interval schedules were arranged with a main key that alternated in color and schedule assignment, along with a changeover key on which a small fixed ratio was required to changeover. Acceptable matching was observed with pigeons in two replications, but there was a tendency toward overmatching. Local response rates were found to differ for unequal schedules of a concurrent pair: local response rate was greater for the variable-interval schedule with the smaller average interreinforcement interval, but qualifications based on an interresponse-time analysis were discussed. In a second experiment, two 3-minute variable-interval schedules were arranged concurrently, and the experimental variable was the changeover procedure: either a changeover delay was incurred by each changeover or a small fixed ratio on a changeover key was required to complete a changeover. Changeover delays of 2 and 5 seconds were compared with a fixed-ratio changeover of five responses. The response output on the main key (associated with the variable-interval schedules) was greater when a changeover delay was arranged than when a fixed ratio was required to changeover. A detailed analysis of stripchart records showed that a 2-second delay generated an increased response rate for 3 seconds after a changeover, while the fixed-ratio requirement generated an increased rate during the first second only, followed by a depressed response rate for 2 seconds.     

Control rate of response or reinforcement and amphetamine's effect on behavior.

The roles of control response rate and reinforcement frequency in producing amphetamine's effect on operant behavior were evaluated independently in rats. Two multiple schedules were arranged in which one variable, either response rate or reinforcement frequency, was held constant and the other variable manipulated. A multiple differential-reinforcement-of-low-rate seven-second yoked variable-interval schedule was used to equate reinforcement frequencies at different control response rates between multiple-schedule components. Amphetamine increased responding under the variable-interval component. In contrast, amphetamine decreased responding equivalently between components of a multiple random-ratio schedule that produced similar control response rates at different reinforcement frequencies. The results provide experimental support to the rate-dependency principle that control rate of responding is an important determinant of amphetamine's effect on operant behavior.     

JEAB: Past,present, and future

The effects of response force on microstructure were evaluated. A strain-gauge operandum permitted the manipulation of the force required to produce reinforcers (criterion responses) independently from the force defining response threshold. Thus, we could detect subcriterion forces that fell short of the force criterion. Eight rats earned food according to variable-interval (VI) 30- and 120-s schedules. The force requirements were set to 5.6 or 32.0 g; the response threshold was fixed at 5.6 g. Interresponse times were computed when subcriterion responses were both included and omitted from the analysis. Log-survivor functions of interresponse times showed that increasing force requirements elevated the mean between-bout interval of the VI 120-s schedule, but only if subcriterion behavior was excluded. Omitting subcriterion responses thus leads to overestimation of intervals separating response bouts. Increasing force requirements also increased the skewness of the between-bout distribution. A subsequent analysis found that subcriterion responses are most plentiful following reinforcer delivery, which helps to explain why their omission might inflate between-bout intervals, as this period is an important transition from reinforcer consumption to engagement in operant activity. The data suggest caution interpreting the effects of force on microstructure when subcriterion behavior is not or cannot be measured.     

Behavior of humans in variable-interval schedules of reinforcement

During Phase I, human subjects pressed a button for monetary reinforcement in five variable-interval schedules, each of which specified a different frequency of reinforcement. The rate of responding was an increasing, negatively accelerated function of reinforcement frequency; the data conformed closely to Herrnstein's equation. During Phase II, the same five schedules were in operation, but in addition a concurrent variable-interval schedule (B) was introduced, responses on which were always reinforced at the same frequency. Response rate in component A increased while the response rate in B decreased, as a function of the reinforcement frequency in component A. Relative response rates in the two component schedules matched the relative frequencies of reinforcement. Comparing the absolute response rates in component A during Phase I and Phase II it was found that introduction of the concurrent schedule did not affect the value of the theoretical maximum response rate, but did increase the value of the reinforcement frequency needed to obtain any particular submaximal response rate.     

Bouts of responding from variable-interval reinforcement of lever pressing by rats

Four rats obtained food pellets by lever pressing. A variable-interval reinforcement schedule assigned reinforcers on average every 2 min during one block of 20 sessions and on average every 8 min during another block. Also, at each variable-interval duration, a block of sessions was conducted with a schedule that imposed a variable-ratio 4 response requirement after each variable interval (i.e., a tandem variable-time variable-ratio 4 schedule). The total rate of lever pressing increased as a function of the rate of reinforcement and as a result of imposing the variable-ratio requirement. Analysis of log survivor plots of interresponse times indicated that lever pressing occurred in bouts that were separated by pauses. Increasing the rate of reinforcement increased total response rate by increasing the rate of initiating bouts and, less reliably, by lengthening bouts. Imposing the variable-ratio component increased response rate mainly by lengthening bouts. This pattern of results is similar to that reported previously with key poking as the response. Also, response rates within bouts were relatively insensitive to either variable.     

Operant conditioning of behavioral variability using a percentile reinforcement schedule.

The present investigation developed and tested a new percentile reinforcement schedule suited to study pattern variability, whose main feature was the relative dissociation it provided between the variability requirement defining criterional responses and overall probability of reinforcement. In a discrete-trials procedure, pigeons produced patterns of four pecks on two response keys. If the pattern emitted on the current trial differed from the N preceding patterns, reinforcement was delivered with probability mu. The schedule continuously adjusted the criterion N such that the probability of a criterional response, estimated from the subject's recent behavior, was always constant. In these circumstances, the criterion corresponded to an invariant percentile in the distribution of recent responses. Using a between-subjects design, Experiment 1 manipulated the variability requirement--the percentile--while keeping overall reinforcement probability constant. The degree of variability varied directly with the requirement. In addition, an inverse relationship existed between the requirement and within-group variance. Experiment 2 manipulated probability of reinforcement while maintaining the variability requirement constant. No consistent relationship was found between variability and reinforcement probability. A tentative hypothesis was advanced ascribing the operant conditioning of behavioral variability to a process of probability-dependent selection.     

Observing responses maintained by conditional discriminative stimuli.

In a conditional discrimination procedure, pigeons' observing responses were analyzed to examine whether two color stimuli (blue or red), conditionally related to whether each of two line stimuli (vertical or horizontal) accompanied reinforcement or nonreinforcement, functioned as conditioned reinforcers. If a variable-interval (VI) 10-s requirement was fulfilled, an observing response produced onset of a color stimulus. A little later, a line stimulus was presented independently of responding, added to the color stimulus to form a compound stimulus. If 55 s elapsed with a response not having occurred either through 55 s or after the variable-interval 10-s had timed out, one of the color-line compound stimuli was presented independently of responding. To control for sensory reinforcement effects and for earlier entrance to the later link, a simple discrimination procedure also was conducted in which reinforcement was not correlated with the color stimuli but with the line stimuli only. As in the conditional discrimination, the observing response also could produce earlier presentation of blue or red. The observing response occurred more frequently during the conditional discrimination than during the simple discrimination. The results were related to different theoretical accounts of conditioned reinforcement, particularly the information hypothesis.     

Relative frequency of reinforcement and rate of punished behavior

After training on a multiple variable-interval variable-interval schedule of reinforcement, each response in one component of the schedule was followed by a brief electric shock. When the rate of punished responses stabilized, the frequency of reinforcement in the other component was first decreased and then increased from the baseline frequency. The effects of these manipulations were consistent with reports of interactions in multiple schedules involving only unpunished behavior, i.e., the rate of punished responses increased with a higher relative frequency of reinforcement in the punishment component and decreased with a lower relative frequency of reinforcement in that component. The relevance of such findings to a further generality of behavioral contrast effects is discussed.     

Escape from an effortful situation

This experiment investigated the tendency to escape from a situation requiring effortful responding. Five human subjects responded in a situation where the response mechanism required 20-lb force to operate; responses were reinforced according to a variable-interval schedule. A subject escaped from this situation by emitting a vocal response which produced a 60-sec “easy period”. During the easy period the reinforcement contingency was switched to a response mechanism requiring 1 lb to operate. It was found that: (1) Escape responding could be conditioned and maintained by producing the easy period; the easy period did not maintain escape responding when the force requirement in the normal situation was equated with it. (2) The rate of escape responding was a function of the magnitude of the force normally required. (3) When easy periods were scheduled after fixed ratios, pausing from the end of the previous easy period to the first escape response was noted. It was concluded that a situation requiring high-force responding is a negative reinforcer. The pattern of fixed-ratio responding suggests that this reinforcer produces typical schedule control in human subjects.     

Performances on ratio and interval schedules of reinforcement: Data and theory

Two differences between ratio and interval performance are well known: (a) Higher rates occur on ratio schedules, and (b) ratio schedules are unable to maintain responding at low rates of reinforcement (ratio “strain”). A third phenomenon, a downturn in response rate at the highest rates of reinforcement, is well documented for ratio schedules and is predicted for interval schedules. Pigeons were exposed to multiple variable-ratio variable-interval schedules in which the intervals generated in the variable-ratio component were programmed in the variable-interval component, thereby “yoking” or approximately matching reinforcement in the two components. The full range of ratio performances was studied, from strained to continuous reinforcement. In addition to the expected phenomena, a new phenomenon was observed: an upturn in variable-interval response rate in the midrange of rates of reinforcement that brought response rates on the two schedules to equality before the downturn at the highest rates of reinforcement. When the average response rate was corrected by eliminating pausing after reinforcement, the downturn in response rate vanished, leaving a strictly monotonic performance curve. This apparent functional independence of the postreinforcement pause and the qualitative shift in response implied by the upturn in variable-interval response rate suggest that theoretical accounts will require thinking of behavior as partitioned among at least three categories, and probably four: postreinforcement activity, other unprogrammed activity, ratio-typical operant behavior, and interval-typical operant behavior.     

Response rate, reinforcement frequency, and behavioral contrast

Pigeons responded for food on a multiple schedule in which periods of green-key illumination alternated with periods of red-key illumination. When behavior had stabilized with a variable-interval 2-min schedule of reinforcement operating during both stimuli, low rates of responding (interresponse times greater than 2 sec) were differentially reinforced during the green component. Conditions during the red stimulus were unchanged. Response rates during the green component fell without changing the frequency of reinforcement but there were no unequivocal contrast effects during the red stimulus. The frequency of reinforcement during the green component was then reduced by changing to a variable-interval 8-min schedule without reducing the response rates in that component, which were held at a low level by the spacing requirement. Again, the conditions during the red stimulus were unchanged but response rates during that stimulus increased. These results show that reductions in reinforcement frequency, independently of response rate, can produce interactions in multiple schedules.