Transfer of persistence to the acquisition of a new behaviour

Two experiments tested whether the degree of effort required for the reinforcement of one behaviour would affect the acquisition of a second behaviour. In the first experiment, rats were placed in a conditioning chamber and: (a) were required to press a lever for food pellets on a fixed ratio schedule, (b) received free presentation of the pellets, or (c) did not receive pellets. Next, all rats were rewarded for a new behaviour, round trips across the length of a runway. As predicted, the fixed-ratio group had the greatest shuttle rate. In the second experiment, two groups were required to press a lever, and the number of presses per pellet was varied. For two other groups not required to press the lever, the amount of food presented per approach to the feeder was varied. The greater required number of lever presses and the lesser number of pellets per approach to the feeder produced the higher subsequent shuttle rates. Two alternative explanations were compared: the degree of accustomed effort per reinforcer becomes a learned component of behaviour, or high effort increases the habituation of frustration-produced disruptive responses.     

Transfer of persistence across behaviors

Two experiments tested the applicability to human beings of findings with animals that the number of performances required for the reinforcement of one behavior affects the subsequent effort expended in other instrumental behaviors. In the first experiment, adult depressed psychiatric patients worked on a sorting task for the approval of a staff psychologist. The time spent and the work completed were increased by prior approval from a ward attendant for each completion of several custodial tasks, as compared to the ward attendant's approval for each completion of a single task or a no-pretreatment control condition. In the second experiment, preadolescent learning-disabled students who were required to read and spell correctly a greater number of words per reward token later spent more time and completed more work for reward tokens in mathematics, and handwriting. Two alternative interpretations of these results are evaluated: (a) The degree of accustomed effort per reinforcer becomes a learned component of behavior, or (b) high effort increases the habituation of frustration-produced disruptive responses. The results suggest that individual differences in general persistence may arise, in part, from an accumulation of effort training in the natural environment.     

JEAB: Past,present, and future

The effects of response force on microstructure were evaluated. A strain-gauge operandum permitted the manipulation of the force required to produce reinforcers (criterion responses) independently from the force defining response threshold. Thus, we could detect subcriterion forces that fell short of the force criterion. Eight rats earned food according to variable-interval (VI) 30- and 120-s schedules. The force requirements were set to 5.6 or 32.0 g; the response threshold was fixed at 5.6 g. Interresponse times were computed when subcriterion responses were both included and omitted from the analysis. Log-survivor functions of interresponse times showed that increasing force requirements elevated the mean between-bout interval of the VI 120-s schedule, but only if subcriterion behavior was excluded. Omitting subcriterion responses thus leads to overestimation of intervals separating response bouts. Increasing force requirements also increased the skewness of the between-bout distribution. A subsequent analysis found that subcriterion responses are most plentiful following reinforcer delivery, which helps to explain why their omission might inflate between-bout intervals, as this period is an important transition from reinforcer consumption to engagement in operant activity. The data suggest caution interpreting the effects of force on microstructure when subcriterion behavior is not or cannot be measured.     

Overshadowing of a stimulus-reinforcer association by an instrumental response

In two experiments, rats were first exposed to pairings of a clicker and food; they were subsequently, in order to measure the effectiveness of the clicker as a conditioned reinforcer, given the opportunity to press a lever which turned the clicker on. For one group of animals the food originally delivered in the presence of the clicker had been contingent on their performance of an instrumental response (running in a running wheel); for a second the contingency between clicker and food had been purely classical. Although the actual correlation between clicker and food was identical for the two groups, the clicker was a less effective conditioned reinforcer for the first group than for the second. In a third experiment, all animals were initially required to run to obtain food in the presence of the clicker, but one group received additional trials on which food was delivered contingent on running in the absence of the clicker. This group showed less tendency to lever press for the clicker than a second group that had received free food on trials when the clicker was not presented. The results of all three experiments suggest that conditioning to the clicker could be overshadowed if the occurrence of food was more reliably predicted by the execution of an instrumental running response; they thus support the view that instrumental conditioning depends on the establishment of an association between response and reinforcer similar to the association between stimulus and reinforcer underlying classical conditioning.     

The form of the auto-shaped response with food or water reinforcers

The relation between the form of auto-shaped responses to the lighting of a key and the consummatory responses of pecking grain and drinking water was examined in pigeons. Responses on the key were analyzed by means of high-speed photography, recordings of the force of contact, and judges' ratings of response-form based on film and videotape recordings. The first experiment showed that food-deprived birds presented grain as a reinforcer responded on the key with a grain-pecking movement, while water-deprived birds presented water as a reinforcer responded with drinking-like movements. The second and third experiments showed that the resemblance between auto-shaped and consummatory responses does not require the dominance of the deprivational state appropriate to the reinforcer. Changing the dominant state of deprivation did not immediately change the form of the key response, and in subjects simultaneously deprived of food and water, the form of response depended on the reinforcer. In the fourth and fifth experiments, subjects simultaneously deprived of food and water received one stimulus signalling food and another signalling water in a random series. In most subjects, the response to each stimulus resembled the consummatory response to the particular reinforcer that was signalled by the stimulus. This result demonstrates the role of association between a stimulus and a reinforcer in producing a resemblance of the auto-shaped response to the consummatory response.     

The effects of differing response-force requirements on fixed-ratio responding of rats.

Rats were exposed to two-component multiple schedules of food delivery. In the first experiment, 15 responses were required to produce food in both components. A downward force of 0.25 N (25 g) was always required to operate the response lever in one component. In the other, the required force was 0.25, 0.50, 1.00, or 2.00 N (25, 50, 100, or 200 g). In the second experiment, 0.25 N of force operated the lever in one component, but in the other, the force requirement for five consecutive responses at the beginning, middle, or end of each ratio was increased from 0.25 to 2.00 N. In the third experiment, the number of responses required to produce food was reduced from 15 to 5, and then to 1. Again, the effects of altering response force from 0.25 to 2.00 N were examined. In general, as response force increased in all experiments, mean response rates decreased and mean interresponse times increased.     

Aversive functions of response effort: Fact or artifact?

Historically, effort has been viewed as aversive. Most supporting evidence comes from studies demonstrating increased force/effort requirements reduce operant responding. Changes in force/effort requirements, however, are often accompanied by changes in response definition when mechanical devices are used to define the response. As a consequence, responses measured at one point in a study may go unmeasured at other points. In an alternative approach, we used a continuous measurement strategy that provided a means to fix the threshold force defining the response class and simultaneously allowed independent manipulation of the force criteria required to produce reinforcement. Rats pressed a force transducer according to a fixed‐ratio 5 schedule of food delivery. The criterion force was systematically increased and decreased; the threshold for response detection was constant. When response rates included only criterion responses, overall rate decreased when force requirements increased. By contrast, when all responses, both those meeting force criteria and those that did not (above the threshold but below the criteria for reinforcement) were included in the rate calculation, increases in force increased response rate. Increases in force criteria also increased the maximum force (g) and time‐integral of force (g‐s) of operant behavior. Control conditions showed increases in responding could be explained by the emergence of subcriterion responses, irrespective of force. We conclude that prior results showing effort decreases response rates are due to an artifact arising from inadvertent changes in response definitions. Increases in effort may better be understood as changes in the response:reinforcer payoff owing to the emergence of a subcriterion response class.     

Feedback functions, optimization, and the relation of response rate to reinforcer rate

The present experiment arranged a series of inverted U-shaped feedback functions relating reinforcer rate to response rate to test whether responding was consistent with an optimization account or with a one-to-one relation of response rate to reinforcer rate such as linear system theory's rate equation or Herrnstein's hyperbola. Reinforcer rate was arranged according to a quadratic equation with a maximum at a unique response rate. The experiment consisted of two phases, during which 6 Long Evans rats lever pressed for food. In the first phase of the experiment, the rats responded on six fixed-interval-plus-quadratic-feedback schedules, and in the second phase the rats responded on three variable-interval-plus-quadratic-feedback schedules. Responding in both phases was inconsistent with a one-to-one relation of response rate to reinforcer rate. Instead, different response rates were obtained at equivalent reinforcer rates. Responding did vary directly with the vertex of the feedback function in both phases, a finding consistent with optimization of reinforcer rate. The present results suggest that the feedback function relating reinforcer rate to response rate imposed by a reinforcement schedule can be an important determinant of behavior. Furthermore, the present experiment illustrates the benefit of arranging feedback functions to investigate assumptions about the variables that control schedule performance.     

Confirmation of linear system theory prediction: Rate of change of Herrnstein's kappa as a function of response-force requirement

Four human subjects worked on all combinations of five variable-interval schedules and five reinforcer magnitudes (¢/reinforcer) in each of two phases of the experiment. In one phase the force requirement on the operandum was low (1 or 11 N) and in the other it was high (25 or 146 N). Estimates of Herrnstein's κ were obtained at each reinforcer magnitude. The results were: (1) response rate was more sensitive to changes in reinforcement rate at the high than at the low force requirement, (2) κ increased from the beginning to the end of the magnitude range for all subjects at both force requirements, (3) the reciprocal of κ was a linear function of the reciprocal of reinforcer magnitude for seven of the eight data sets, and (4) the rate of change of κ was greater at the high than at the low force requirement by an order of magnitude or more. The second and third findings confirm predictions made by linear system theory, and replicate the results of an earlier experiment (McDowell & Wood, 1984). The fourth finding confirms a further prediction of the theory and supports the theory's interpretation of conflicting data on the constancy of Herrnstein's κ.     

Instrumental responding following reinforcer devaluation

In two experiments, hungry rats were given instrumental lever-press training for an appetitive reinforcer and, in addition, were exposed to another type of food which was not contingent on lever pressing. In the first experiment, exposure to each type of food was on separate days, whereas in the second experiment rats were exposed to each type of food in strict alternation within each session. Subsequently, a food aversion was conditioned to the reinforcer for the experimental group and to the non-contingent food for the control group. In both experiments, animals with an aversion to the reinforcer responded less in an extinction test than animals with an aversion to the non-contingent food. Subsequent reacquisition tests confirmed that the aversion to the non-contingent food in the control group was of comparable strength with that to the reinforcer in the experimental group. The results were discussed in terms of whether the reinforcer is encoded in the associative structure set up by exposure to an instrumental contingency.     

The role of instrumental responding and contiguity of stimuli in the development of infant secondary reinforcement

Ten-month-old infants received contingent pairings of a tone (T+) and food reinforcer. Groups S^r and S^D received the food on an FI 23-sec schedule for target touching, the former group receiving T+ immediately after the response and 1.5 sec prior to food and the latter group receiving T+ at the end of the intertrial interval. Group SC received food reinforcers 1.5 sec after T+ with no response required. A second tone (Tⁿ) was heard by all groups once during each intertrial interval, at randomly determined points. All groups subsequently were given a spatial discrimination task, receiving T+ for one alternative and Tⁿ for the other. Group S^r gave significantly more responses for T+ than for Tⁿ, but neither of the other two groups produced a superiority for T+. Thus, both contiguity with a primary reinforcer and the presence of an operant during training appear to be necessary for a neutral signal to acquire the ability to enhance responding.     

Preference and response suppression under different correlations between shock and a positive reinforcer in rats

A lever-press response by rats was reinforced by food in two successively presented types of trial signalled by different discriminative stimuli. When responding was punished by a shock in one type of trial, a groups for which the shock always preceded the reinforcer by .5 sec (positive correlation) showed less suppression in those trials than a group which received the shock and food separated in time (negative correlation). When, in a second experiment, rats were given a choice between food alone in one end of a shuttle box and either positively or negatively correlated shock and food in the other on a concurrent schedule, the group receiving the negatively correlated shock showed a greater preference for the food alone end. On the basis of a third experiment in which a tone was substituted for the shock in the choice situation, it was argued that the effect of correlation was not simply due to the stimulus properties of the shock. A final experiment demonstrated that when shock punishment is administered during extinction of the lever-press response, the rate of extinction is slower if the shock has been previously paired with the food reinforcer. Pairing a shock with food seems to attenuate the intrinsic aversiveness of the shock through Pavlovian counterconditioning.     

Effects of pre-trial response requirements on self-control choices by rats and pigeons

Parallel experiments with rats and pigeons examined whether the size of a pre-trial ratio requirement would affect choices in a self-control situation. In different conditions, either 1 response or 40 responses were required before each trial. In the first half of each experiment, an adjusting-ratio schedule was used, in which subjects could choose a fixed-ratio schedule leading to a small reinforcer, or an adjusting-ratio schedule leading to a larger reinforcer. The size of the adjusting ratio requirement was increased and decreased over trials based on the subject's responses, in order to estimate an indifference point-a ratio at which the two alternatives were chosen about equally often. The second half of each experiment used an adjusting-delay procedure-fixed and adjusting delays to the small and large reinforcers were used instead of ratio requirements. In some conditions, particularly with the reinforcer delays, the rats had consistently longer adjusting delays with the larger pre-trial ratios, reflecting a greater tendency to choose the larger, delayed reinforcer when more responding was required to reach the choice point. No consistent effects of the pre-trial ratio were found for the pigeons in any of the conditions. These results may indicate that rats are more sensitive to the long-term reinforcement rates of the two alternatives, or they may result from a shallower temporal discounting rate for rats than for pigeons, a difference that has been observed in previous studies.     

Maintenance-feeding effort affects instrumental performance

The effort required of rats to obtain food, employing standard maintenance-feeding procedures, can affect the effort subsequently expended in an instrumental-learning task. Rats received 43 food-rewarded runway trials followed by access to food (a) on the home-cage floor, or (b) from a hopper attached to the home-cage's wiremesh front wall. Hopper feeding involved the greater effort since the rats had to gnaw the food pellets through the wire mesh. Following 9 or 27 days of maintenance feeding, the rats were returned to the runway for one rewarded trial and 16 extinction trials. The hopper-fed rats ran faster than floor-fed rats on the rewarded test trial. In extinction, long-term maintenance feeding produced faster subsequent running times by the hopper-fed rats than the floor-fed rats. Short-term maintenance feeding, on the other hand, produced slower running times by the hopper-fed rats than the floor-fed rats. The results are consistent with previous findings of transfer of effort across behaviours and suggest that the effort required for reinforcement in the maintenance environment can systematically influence the effectiveness of various experimental treatments.     

Three factors that promote positive induction when rats respond for 1% sucrose

Studies have demonstrated that rats will increase their operant rate of response for a low-valued reinforcer if a high-valued reinforcer will be available later in the session. Research on this positive induction effect suggests that at least three factors account for its appearance: premature responding for the yet unavailable high-valued reinforcer, an increase in the reinforcing value of the low-valued reinforcer, and responding controlled (e.g., elicited) by the response manipulandum. The present experiment tested whether the size of induction could be systematically altered by varying these factors. Twenty-four rats responded in sessions in which 1% sucrose or a food pellet served as the reinforcer in the first or second half of the session. In some sessions, the same operant response was required in both halves of the session. In others, different responses were required. Half of the rats received the different reinforcers in one food trough while the other half received reinforcers in the different halves of the session in different food troughs. Results demonstrated that a large positive induction effect was observed when all of the above factors were present to contribute to the effect (i.e., high-valued reinforcer upcoming, earned by making the same response, delivered to the same food trough). A small, but significant, induction effect remained when all three were absent (i.e., high-valued reinforcer delivered first, earned by making a different response, delivered to a different food trough). The results support the idea that these three factors are the main contributors to the appearance of this positive induction effect. However, at least one additional factor must also contribute.     

Effects of response-allocation constraints on multiple-schedule performance

Four pigeons were trained on multiple variable-interval schedules in which components alternated after a fixed number of responses had been emitted. In Part 1, each component change occurred after 20 responses; in Part 2, the number was 40; and in Part 3, the number of responses before change was 10. Component reinforcer rates were varied over five experimental conditions in each of Parts 1 to 3. Component response rates decreased as the specified number of responses per component was increased. However, the relation between component response-rate ratios and component reinforcer-rate ratios was independent of the specified number of responses per component, and was similar to that found when components alternate after fixed time periods. In the fourth part of the experiment, the results from Parts 1 to 3 were systematically replicated by keeping the component reinforcer rates constant, but different, while the number of responses that produced component alternation was varied from 5 to 60 responses. The results showed that multiple-schedule performance under component-response-number constraint is similar to that under conventional component-duration constraint. They further suggest that multiple-schedule response rates are controlled by component reinforcer rates and not by principles of maximizing overall reinforcer rates or meliorating component reinforcer rates.     

Within-trial contrast: pigeons prefer conditioned reinforcers that follow a relatively more rather than a less aversive event

When behavior suggests that the value of a reinforcer depends inversely on the value of the events that precede or follow it, the behavior has been described as a contrast effect. Three major forms of contrast have been studied: incentive contrast, in which a downward (or upward) shift in the magnitude of reinforcement produces a relatively stronger downward (or upward) shift in the vigor of a response; anticipatory contrast, in which a forthcoming improvement in reinforcement results in a relative reduction in consummatory response; and behavioral contrast, in which a decrease in the probability of reinforcement in one component of a multiple schedule results in an increase in responding in an unchanged component of the schedule. Here we discuss a possible fourth kind of contrast that we call within-trial contrast because within a discrete trial, the relative value of an event has an inverse effect on the relative value of the reinforcer that follows. We show that greater effort, longer delay to reinforcement, or the absence of food all result in an increase in the preference for positive discriminative stimuli that follow (relative to less effort, shorter delay, or the presence of food). We further distinguish this within-trial contrast effect from the effects of delay reduction. A general model of this form of contrast is proposed in which the value of a primary or conditioned reinforcer depends on the change in value from the value of the event that precedes it.     

Are conditioned stimuli determinants of oriented exploratory behavior in the rat?

To test Bindra and Palfai's hypothesis about enhancing and inhibitory motivational effects of a conditioned stimulus on exploratory behavior, rats in a first experiment were submitted to pairings of a tone with (a) water, (b) intermittent deliveries of water after a period of constant pairing. In a control condition, the tone was presented alone. The tone was subsequently presented during oriented exploration of a novel stimulus, in the conditioning context. There was no difference in exploratory performance between conditions. In a second experiment with another kind of stimulus change and a larger number of training trials, exploration did not increase when occurring during CS presentation. The lack of facilitation which is possibly the product of competition between exploration and magazine-directed responses, can also be interpreted as indicating that the effects of a conditioned stimulus are reinforcer specific and that the diffuse activity it generates in a situation different from the training one is not equivalent, as concerns the processes involved, to oriented exploratory behavior.     

Effects of differing response-force requirements on food-maintained responding in CD-1 mice

The effect of force requirements on response effort was examined using outbred (CD-1) mice trained to press a disk with their snout. Lateral peak forces greater than 2 g were defined as threshold responses (i.e., all measured responses). Different force requirements were used to define criterion responses (a subclass of threshold responses) that exceeded the requirement. The reinforcer was sweetened, condensed milk, and it was delivered upon response termination. All mice were exposed to two ascending series of criterion force requirements (2, 4, 8, 16, and 32 g). Increasing the force requirement decreased criterion response rates, but increased threshold response rates. The time-integral of force (area under the force-time curve for individual responses, which is proportional to energy expenditure for each response) increased with the increase in the force requirement. These results conflict with the hypothesis that higher force requirements have aversive qualities and suggest that increased force requirements are more analogous to intermittent schedules of reinforcement. These data suggest that estimations of effort or energy expenditure should be measured independently of the force requirement. Individual differences in responding were found for the CD-1 outbred stock.     

Resurgence and alternative‐reinforcer magnitude

Resurgence is defined as an increase in the frequency of a previously reinforced target response when an alternative source of reinforcement is suspended. Despite an extensive body of research examining factors that affect resurgence, the effects of alternative‐reinforcer magnitude have not been examined. Thus, the present experiments aimed to fill this gap in the literature. In Experiment 1, rats pressed levers for single‐pellet reinforcers during Phase 1. In Phase 2, target‐lever pressing was extinguished, and alternative‐lever pressing produced either five‐pellet, one‐pellet, or no alternative reinforcement. In Phase 3, alternative reinforcement was suspended to test for resurgence. Five‐pellet alternative reinforcement produced faster elimination and greater resurgence of target‐lever pressing than one‐pellet alternative reinforcement. In Experiment 2, effects of decreasing alternative‐reinforcer magnitude on resurgence were examined. Rats pressed levers and pulled chains for six‐pellet reinforcers during Phases 1 and 2, respectively. In Phase 3, alternative reinforcement was decreased to three pellets for one group, one pellet for a second group, and suspended altogether for a third group. Shifting from six‐pellet to one‐pellet alternative reinforcement produced as much resurgence as suspending alternative reinforcement altogether, while shifting from six pellets to three pellets did not produce resurgence. These results suggest that alternative‐reinforcer magnitude has effects on elimination and resurgence of target behavior that are similar to those of alternative‐reinforcer rate. Thus, both suppression of target behavior during alternative reinforcement and resurgence when conditions of alternative reinforcement are altered may be related to variables that affect the value of the alternative‐reinforcement source.