Aversive functions of response effort: Fact or artifact?

Historically, effort has been viewed as aversive. Most supporting evidence comes from studies demonstrating increased force/effort requirements reduce operant responding. Changes in force/effort requirements, however, are often accompanied by changes in response definition when mechanical devices are used to define the response. As a consequence, responses measured at one point in a study may go unmeasured at other points. In an alternative approach, we used a continuous measurement strategy that provided a means to fix the threshold force defining the response class and simultaneously allowed independent manipulation of the force criteria required to produce reinforcement. Rats pressed a force transducer according to a fixed‐ratio 5 schedule of food delivery. The criterion force was systematically increased and decreased; the threshold for response detection was constant. When response rates included only criterion responses, overall rate decreased when force requirements increased. By contrast, when all responses, both those meeting force criteria and those that did not (above the threshold but below the criteria for reinforcement) were included in the rate calculation, increases in force increased response rate. Increases in force criteria also increased the maximum force (g) and time‐integral of force (g‐s) of operant behavior. Control conditions showed increases in responding could be explained by the emergence of subcriterion responses, irrespective of force. We conclude that prior results showing effort decreases response rates are due to an artifact arising from inadvertent changes in response definitions. Increases in effort may better be understood as changes in the response:reinforcer payoff owing to the emergence of a subcriterion response class.     

Operant conditioning of autogrooming in vervet monkeys: Cercopithecus aethiops

Vervet monkeys received food reinforcement contingent on autogrooming. Experiment 1 reinforced grooming on a schedule of increasing intermittency and grooming increased in frequency and duration; with only pauses reinforced, grooming decreased in frequency and duration. Experiment 2 demonstrated differentiation of operant autogrooming; in each session a different single form of grooming was reinforced (for example, grooming the tail only), and that form increased in frequency while other forms became less frequent. In Experiment 3 scratching was succesfully conditioned with a method that selectively reinforced variety in behavior; reinforcement was contingent on a shift in scratching form. In Experiment 4, with no contingencies on grooming, a prefood stimulus did not increase autogrooming whether or not grooming had previously resulted in contingent reinforcement. The form of conditioned autogrooming resembled the form of unconditioned autogrooming. The discussion suggests how reinforcement principles can account for changes in the topography of operant behavior.     

Differential antagonism of cocaine self-administration and cocaine-induced disruptions of learning by haloperidol in rhesus monkeys

Six rhesus monkeys responding under a three-component multiple schedule were administered haloperidol to determine its effects on cocaine self-administration and on cocaine's disruptive effects on the repeated acquisition and performance of response chains. In the absence of haloperidol, 0.0032-0.032 mg/kg/infusion of cocaine increased response rate and the number of infusions in the self-administration component when compared to saline administration, whereas 0.1-0.32 mg/kg/infusion decreased response rate and the number of infusions. When compared to saline administration, the two lowest infusion doses of cocaine had little or no effect on responding in the acquisition and performance components; however, higher infusion doses of cocaine dose-dependently decreased response rate in these components. In addition, the higher doses of cocaine also increased the percentage of errors in the acquisition and performance components. Pretreatment with haloperidol (0.0032 or 0.01 mg/kg, i.m.) antagonized the effects of low doses of cocaine on the number of infusions in the self-administration component, whereas only the 0.01-mg/kg dose antagonized the effects of high doses of cocaine on the number of infusions. Neither dose of haloperidol antagonized the rate-decreasing effects of cocaine on responding in the acquisition and performance components significantly; the highest dose of haloperidol alone decreased rates of responding in each component. Antagonism of cocaine's error-increasing effects by haloperidol was only evident at one dose of cocaine (0.032 mg/kg/infusion), and was more complete in the performance components than in the acquisition components. Together, these data show the limited suitability of haloperidol for selectively antagonizing cocaine self-administration in the context of a multiple schedule involving transition behavior, and show the lack of uniform antagonism across operant behaviors.     

Differentiation of a precise timing response

Humans, monkeys, and rats were trained by a process of successive differentiations to press a bar for at least 1.00 sec but for no longer than 1.27 sec. Initially, animals were reinforced for all responses, then a minimum duration of response was gradually differentiated, below which no responses were reinforced. Finally, a maximum duration of response was differentiated above which no responses were reinforced. The duration of response in all three species approximated the minimum duration of response necessary for reinforcement. As the duration of response necessary for reinforcement increased, so did the mean duration of response in the three species. As the maximum allowable duration decreased, further compression of the mean occurred. The fact that the acquisition of the differentiation was approximately the same in all three species is a further indication of the control reinforcement exerts on operant responding.     

A minicomputer system for recording the dynamic properties of individual operant responses

A PDP-12 is used to conduct operant experiments in which peak force, duration, time integral of force, and interresponse time may each serve as the criterion response property for reinforcement, and all four properties are simultaneously recorded as dependent variables. Calibration, acquisition and control, and data analysis programs are described.     

Resistance to change of operant variation and repetition

A multiple chained schedule was used to compare the relative resistance to change of variable and fixed four-peck response sequences in pigeons. In one terminal link, a response sequence produced food only if it occurred infrequently relative to 15 other response sequences (vary). In the other terminal link, a single response sequence produced food (repeat). Identical variable-interval schedules operated in the initial links. During baseline, lower response rates generally occurred in the vary initial link, and similar response and reinforcement rates occurred in each terminal link. Resistance of responding to prefeeding and three rates of response-independent food delivered during the intercomponent intervals then was compared between components. During each disruption condition, initial- and terminal-link response rates generally were more resistant in the vary component than in the repeat component. During the response-independent food conditions, terminal-link response rates were more resistant than initial-link response rates in each component, but this did not occur during prefeeding. Variation (in vary) and repetition (in repeat) both decreased during the response-independent food conditions in the respective components, but with relatively greater disruption in repeat. These results extend earlier findings demonstrating that operant variation is more resistant to disruption than is operant repetition and suggest that theories of response strength, such as behavioral momentum theory, must consider factors other than reinforcement rate. The implications of the results for understanding operant response classes are discussed.     

The basis of behavioral momentum in the nonlinearity of strength

The persistence of operant responding in the context of distractors and opposing forces is of central importance to the success of behavioral interventions. It has been successfully analyzed with Behavioral Momentum Theory. Key data from the research inspired by that theory are reanalyzed in terms of more molecular behavioral mechanisms: the demotivational effects of disruptors, and their differential impacts on the target response and other responses that interact with them. Behavioral momentum is regrounded as a nonlinear effect of motivation and reinforcement rate on response probability and persistence. When response probabilities are high, more energy is required to further increase or to decrease them than when they are low. Classic Behavioral Momentum Theory effects are reproduced with this account. Finally, it is shown how the new account involving motivation and competition is closely related to the metaphor of force and action that is at the core of Behavioral Momentum Theory.     

Quantification of rats' behavior during reinforcement periods

What is treated as a single unit of reinforcement often involves what could be called a reinforcement period during which two or more acts of ingestion may occur, and each of these may have associated with it a series of responses, some reflexive, some learned, that lead up to ingestion. Food-tray presentation to a pigeon is an example of such a “reinforcement period.” In order to quantify this behavior, a continuous-reinforcement schedule was used as the reinforcement period and was chained to a fixed-ratio schedule. Both fixed-ratio size and reinforcement-period duration were manipulated. Rats were used as subjects, food as reinforcement, and a lever press as the operant. Major findings included (a) a rapid decline in response rates during the first 15 to 20 seconds of the reinforcement periods, and (b) a strong positive relationship between these response rates and the size of the fixed ratio. Also revealed was a short scallop not normally found in fixed-ratio response patterns, whose length was a function of fixed-ratio size and reinforcement-period duration. It is suggested that rapidly fluctuating excitatory processes can account for many of these findings and that such processes are functionally significant in terms of behavioral compensation.     

Force and rate relations in responding during variable-interval reinforcement

Four rats responded on one-minute variable-interval schedules with several variations in peak-force of response required for food reinforcement. Measures of peak force and rate were taken for the responses, which were the downward exertions of force against a static force-transducing operandum. The analysis distinguished responses, a generic class of measured behavior, from criterion responses, an operationally specified subclass required for reinforcement. Absolute rate of response showed no systematic change, but the rate of responses meeting a newly required criterion of peak-force invariably increased through changes in the absolute rate of response, the relative-frequency distributions of peak force, or some combination of both. The relative frequency of responses meeting an elevated force criterion during variable-interval reinforcement exceeded that maintained with the same criterion with continuous reinforcement. The requirement of more effortful responding for reinforcement does not necessarily reduce response rate. Conformity of the behavior to the requirement for reinforcement is the salient effect.     

A CLASSIFICATION SCHEME OF SOCIAL CONDITIONS OF REINFORCEMENT WITHIN GROUP OPERANT SYSTEMS1

Many different reinforcement contingencies are found in group operant systems, such as token economies and point systems. Some systems use group contingencies in which the reinforcement of any one participant may depend on the behavior of some other group member. Other programs are individual, in that participants earn reinforcers dependent only on their own behavior. The various possible arrangements of people and their response requirements are labelled “social conditions of reinforcement” in this paper. Previous attempts at classification have failed to categorize the variety of social conditions of reinforcement. In addition, some conditions that may produce behaviorally different effects have not been separated. The present paper classifies the social conditions of reinforcement found in applied programs in a three-dimensional scheme. The efficacy of the three major dimensions—reinforcing agent, recipient response requirement, and group response requirement—is supported by clinical and research data. The reinforcing agent dimension refers to the person(s) who dispenses reinforcers to group members. This major dimension is further subdivided: one or several agents may be either designated or nondesignated. Recipients are the group members who receive reinforcement. This dimension is also subdivided: one or several recipients in a social condition of reinforcement may obtain reinforcers either contingently or noncontingently. The group response requirement is a criterion that must be satisfied before any group participant is eligible for reinforcement. Some systems have no group requirement, and others have a group requirement that must be met by some designated or nondesignated participant(s). Supportive references and examples are given in the explanation of each dimension and subdimension. The behavioral impact of the various categories is emphasized. For all major dimensions, applied implications and research suggestions are discussed. Concluding remarks center on the utility of the present scheme, the classification of operant procedures other than positive reinforcement, and both theoretical and applied issues requiring further study (e.g., the long-term effects of participation in group contingencies).     

Response-dependent point loss and response force as disrupting operations on behavioral resistance to change in humans

Behavioral momentum theory (BMT) provides a theoretical and methodological framework for understanding how differentially maintained operant responding resists disruption. A common way to test operant resistance involves contingencies with suppressive effects, such as extinction or prefeeding. Other contingencies with known suppressive effects, such as response-cost procedures arranged as point-loss or increases in response force, remain untested as disruptive events within the BMT framework. In the present set of three experiments, responding of humans was maintained by point accumulation programmed according to a multiple variable-interval (VI) VI schedule with different reinforcement rates in either of two components. Subsequently, subtracting a point following each response (Experiment 1) or increasing the force required for the response to be registered (Experiments 2 and 3 decreased response rates, but responding was less disrupted in the component associated with the higher reinforcement rate. The point-loss contingency and increased response force similarly affected response rates by suppressing responding and human persistence, replicating previous findings with humans and nonhuman animals when other types of disruptive events (e.g., extinction and prefeeding) were investigated. The present findings moreover extend the generality of the effects of reinforcement rate on persistence, and thus BMT, extending the analysis of resistance to two well-known manipulations used to reduce responding in the experimental analysis of behavior.     

Changes in feeding and foraging patterns as an antipredator defensive strategy: a laboratory simulation using aversive stimulation in a closed economy.

The effects of the risk of electric shock on the meal patterns of rats living in an operant chamber were investigated. Rats could obtain food by working on a response lever that provided reinforcement according to chained fixed-ratio continuous reinforcement schedules that allowed the animals control over meal size. Using a two-compartment operant chamber with a safe nesting area and manipulanda area with a grid floor, shock could be correlated with responding on the schedule. Shocks (less than or equal to 1.25 per hour) were scheduled to occur randomly throughout the day, independent of the rat's behavior. Shock caused a reorganization of meal patterns such that the animals took less frequent but larger meals. This pattern reduced the time the animals spent at risk without compromising caloric balance. Similar changes in feeding pattern were obtained in both hooded and albino rats. Exposure to shock in a separate chamber did not produce these behavioral modifications. The magnitude of shock-induced alterations of meal patterns was greater with chained fixed-ratio 90 continuous reinforcement than with chained fixed-ratio 10 continuous reinforcement. Additionally, the rats seemed to be able to reduce food intake but increase caloric efficiency, such that the reduced food intake did not have deleterious effects on maintenance of body weight. These behavioral modifications reduced the number of shocks received from that which would have been expected if meal pattern changes had not occurred. We suggest that this technique may provide a useful laboratory simulation of the impact that the risk of predation has on foraging behavior.     

Independence of response force and reinforcement rate on concurrent variable-interval schedule performance

Five pigeons were trained over 43 experimental conditions on a variety of concurrent variable-interval schedules on which the forces required on the response keys were varied. The results were well described by the generalized matching law with log reinforcement ratios and log force ratios exerting independent (noninteractive) effects on preference. A further analysis using the Akaike criterion, an information-theoretic measure of the efficiency of a model, showed that overall reinforcement rate and overall force requirement did not affect preference. Unlike reinforcement rate changes, force requirement increases did not change the response rate on the alternate key, and an extension of Herrnstein's absolute response rate function for force variation on a single variable-interval schedule is suggested.     

Early-extinction effects on a force response of humans

Humans were used to investigate changes in response force occurring soon after reinforcement was eliminated. In Experiment 1, in a 300-s baseline phase, 10 participants received a point for holding down a pressure sensor set to operate at a force equal to 85% of the maximum force the participants exerted during a pretest. Following this, during a 600-s extinction phase, criterion responses had no consequence. In Experiment 2, 6 participants worked on the same task, but (a) points were exchangeable for money and (b) after extinction, the reinforcement baseline phase was reinstated. In Experiment 3, 6 participants completed the same task as in Experiment 2, but the required minimum force was 60% of the maximum force exerted during the pretest. In each experiment, increases in response force relative to the mean and peak force exerted during the last 100 s of baseline were observed in most participants when force responses were aggregated into short sample intervals, but less so with longer ones. The increases, however, were not systematic across or within participants, questioning the generality of and the criteria for demonstrating an extinction burst.     

The sensitivity of response rate to the rate of variable-interval reinforcement for pigeons and rats: a review

The relation between the rate of a response (B) and the rate of its reinforcement (R) is well known to be approximately hyperbolic: B = kR/(R + Ro), where k represents the maximum response rate, and Ro indicates the rate of reinforcers that will engender a response rate equal to half its maximum value. A review of data reported in 17 published papers revealed that, under variable-interval schedules of reinforcement, Ro was usually lower when pigeons were the subjects than when rats were the subjects. The value of k, in contrast, did not differ consistently between pigeons and rats. Some accounts interpret Ro as the rate of alternative, unscheduled reinforcers in the situation, expressed in units of the scheduled reinforcer. So interpreted, the difference in Ro implies that less alternative reinforcement (relative to the scheduled reinforcement) typically is available to pigeons in their operant conditioning chambers than it is to rats in theirs. Whether or not that interpretation of Ro is valid, the pigeon-rat difference in Ro ensures that for reinforcer rates above about 10 per hour, response rate will be noticeably less sensitive to changes in reinforcer rate (and presumably to changes in other incentive and motivational operations) with pigeons than with rats as subjects, at least with the experimental conditions typically employed.     

Changes in energy expenditure and work during response acquisition in rats

The principle of least effort predicts that behavior will tend to maximum efficiency. To test this prediction, changes in the energy expended (VO2) and work performed per reinforcement were monitored continuously as rats learned to press a beam with a criterion force for liquid food rewards. All 12 subjects exhibited significant decreases in energy expended per reinforcement over the 16 days of observation. Of these, 10 subjects also decreased the work performed per reinforcement. Analyses of motor performance were undertaken to determine how motor programs for changing efficiency were generated. The 10 animals showing decreased work reinforcement also exhibited significant decreases in the variability of temporal and kinetic response features and in mean response magnitude (time integral of force or work per response) as a function of practice. Adjustments in work output were primarily accomplished by modifying temporal response features (response duration and, initially, interresponse time). The kinetic features (response recruitment and peak force) remained relatively constant for these animals. The remaining 2 subjects differed in that response recruitment increased after Day 9, resulting in progressively larger amounts of work being performed to earn each reinforcement, and the interval between successive reinforcements decreased.     

Response-initiated imaging of operant behavior using a digital camera

A miniature digital camera, QuickCam Pro 3000, intended for use with video e-mail, was modified so that snapshots were triggered by operant behavior emitted in a standard experimental chamber. With only minor modification, the manual shutter button on the camera was replaced with a simple switch closure via an I/O interface controlled by a PC computer. When the operant behavior activated the I/O switch, the camera took a snapshot of the subject's behavior at that moment. To illustrate the use of the camera, a simple experiment was designed to examine stereotypy and variability in topography of operant behavior under continuous reinforcement and extinction in 6 rats using food pellets as reinforcement. When a rat operated an omnidirectional pole suspended from the ceiling, it also took a picture of the topography of its own behavior at that moment. In a single session after shaping of pole movement (if necessary), blocks of continuous reinforcement, in which each response was reinforced, alternated with blocks of extinction (no reinforcement), with each block ending when 20 responses had occurred. The software supplied with the camera automatically stored each image and named image files successively within a session. The software that controlled the experiment also stored quantitative data regarding the operant behavior such as consecutive order, temporal location within the session, and response duration. This paper describes how the two data types--image information and numerical performance characteristics-can be combined for visual analysis. The experiment illustrates in images how response topography changes during shaping of pole movement, how response topography quickly becomes highly stereotyped during continuous reinforcement, and how response variability increases during extinction. The method of storing digital response-initiated snapshots should be useful for a variety of experimental situations that are intended to examine behavior change and topography.     

Constituents of response rates

Response rate and the proportion of time pigeons allocated to a key-pecking activity were measured on several basic types of reinforcement schedules. Reinforcement frequency was varied within each type of basic schedule, and the effects on two constituents of response rate were noted. Propensity, the proportion of time the birds spent on a platform in front of the key, showed very consistent effects as reinforcement frequency varied: in general, it decreased when reinforcement frequency markedly decreased and it increased when reinforcement frequency increased. Speed, key pecks per unit of time spent on the platform, showed inconsistent effects when reinforcement frequency varied. Consequently, response rate showed less consistent effects than did propensity. Cumulative response records demonstrated the existence of several different types of transitions or boundary states between the key-pecking activity and other activities. The types of transitions that occurred between activities depended on both the type of reinforcement schedule and the frequency of reinforcement. The propensity data support the position that general laws of behavior can be based on temporal measures of behavior. The speed data suggest that, if a complete assessment of the dynamic properties of behavior is to be achieved, measures of behavior must incorporate the structural variations in the operant unit.     

Concurrent second-order schedules: some effects of variations in response number and duration

To examine the effects on concurrent performance of independent manipulations of response-unit duration and number, 6 hens were exposed to concurrent second-order schedules of reinforcement. Each first-order operant unit required completion of a fixed-ratio schedule within the time specified by a fixed-interval schedule, with one further response completing the fixed-interval schedule. The fixed-ratio and fixed-interval requirements comprising the first-order operant units were systematically and independently varied under three pairs of concurrent variable-interval schedules to produce differences in the first-order response and duration requirements (response and duration differentials). These manipulations produced consistent changes in response, time, and operant-unit biases. A 1:4 response differential biased the time and operant-unit measures towards the smaller fixed ratio, but to a degree less than the imposed response differential. The response-based biases favored the larger fixed ratio. Duration differentials of 4:1 and 8:1 biased the response and operant-unit measures towards the shorter fixed interval, again less than the imposed duration differential, but the time biases remained close to zero. Both sorts of differentials acted to bias operant-unit completions more systematically than the other measures, but undermatching to the differentials occurred. The undermatching appears to have arisen from a pattern of fix and sample (in which visits to the less preferred alternative involved only a single completed operant unit) under combinations of unequal operant-unit requirements and reinforcer rates. The response and time bias measures appeared to arise as by-products of the changes in operant-unit completions.     

Minimized tolerance to the suppressive effects of CCK8 on operant responding

This study was designed to determine if tolerance to cholecystokinin octapeptide (CCK8) could be prevented or minimized by spacing injections, and if an appetitive operant conditioning paradigm provides a sensitive baseline to test the effects of CCK8. Male Sprague-Dawley rats were trained to press a lever for water according to a fixed-interval 60-sec schedule of reinforcement. After response rate stabilized, rats were given a series of CCK8 (0.04 mg/kg, ip) injections spaced several days apart. The first injection of CCK8 produced complete response suppression during the 30-min test session, while later injections produced partial or complete suppression. Thus, tolerance to CCK8, as measured by operant response rate, may be minimized if injections are appropriately spaced.