Uninstructed human responding: sensitivity to ratio and interval contingencies

College students' presses on a telegraph key were occasionally reinforced by light onsets in the presence of which button presses (consummatory responses) produced points later exchangeable for money. One student's key presses were reinforced according to a variable-ratio schedule; key presses of another student in a separate room were reinforced according to a variable-interval schedule yoked to the interreinforcement intervals produced by the first student. Instructions described the operation of the reinforcement button, but did not mention the telegraph key; instead, key pressing was established by shaping. Performances were comparable to those of infrahuman organisms: variable-ratio key-pressing rates were higher than yoked variable-interval rates. With some yoked pairs, schedule effects occurred so rapidly that rate reversals produced by schedule reversals were demonstrable within one session. But sensitivity to these contingencies was not reliably obtained with other pairs for whom an experimenter demonstrated key pressing or for whom the reinforcer included automatic point deliveries instead of points produced by button presses. A second experiment with uninstructed responding demonstrated sensitivity to fixed-interval contingencies. These findings clarify prior failures to demonstrate human sensitivity to schedule contingencies: human responding is maximally sensitive to these contingencies when instructions are minimized and the reinforcer requires a consummatory response.     

A behavioral economic analysis of concurrently available money and cigarettes.

In economic terms, consumption of a reinforcer is determined by its price and the availability and price of other reinforcers. This study examined the effects of response-requirement (i.e., price) manipulations on the self-administration of two concurrently available reinforcers. Six cigarette smokers participated in 4-hr sessions in which money and puffs on a cigarette were concurrently available according to fixed-ratio schedules of reinforcement. Once stable responding was obtained with both reinforcers available at Fixed Ratio 100, the response requirement for one reinforcer was systematically varied (Fixed Ratio 1,000 and 2,500), while the other reinforcer remained scheduled at Fixed Ratio 100. Increasing the fixed-ratio size for a reinforcer decreased its consumption, with a greater decrease occurring for monetary reinforcement. This finding was quantified in economic terms as own-price elasticity, with elasticity coefficients greater for money than cigarettes. The effects of fixed-ratio size on response output also differed across the two reinforcers. Although greater responding occurred for money at Fixed Ratio 100, increases in fixed-ratio size (for money) decreased responding for money, whereas the same increase in fixed-ratio size (for puffs) increased responding for puffs. Finally, increasing the fixed-ratio size for one reinforcer had little effect on consumption of the other concurrently available reinforcer. This finding was quantified as cross-price elasticity, with elasticity coefficients near 0.0 for most subjects, indicating little or no reinforcer interaction. The results indicate that the reinforcing effects of cigarettes and money in the setting studied here differed, and that the effects produced by changing the price of one reinforcer did not interact with the consumption of the other concurrently available reinforcer.     

Reciprocal cooperation in dyads

This study investigated the conditions that promote long-term social exchange when the immediate payoff for social participation is equal to or less than for individual responding. The 10 subjects, from a special education unit and ranging between 10 and 16 years of age, comprised five dyads to participate in daily 20-min sessions. During the first experiment, the subjects sat beside each other and in front of an automated apparatus which automatically distributed points (representing money) to self, to partner, or to the group. During baseline conditions, the subjects could earn reinforcers from all three modes. During the dependency conditions, which alternated between subjects at 3-day intervals, one subject could earn reinforcers from all three modes while a partner could earn reinforcers only from the mode requiring subject's assistance. Four of the five dyads increased their levels of cooperative responding during the several reversals in which first one and then the other subject was dependent upon the partner for reinforcement. This pattern maintained when the method of distributing reinforcers was changed to a manual procedure requiring the experimenter to record point distributions by transferring beads on an abacus. This suggested that the distribution modes rather than the experimental apparatus were important in promoting the cooperative pattern. In the second experiment all subjects continued to cooperate with partner, even when they could have earned more by working alone. Eventually a value for the self mode was reached where the subjects discontinued their cooperation to work for self.     

Instructed versus shaped human verbal behavior: Interactions with nonverbal responding

Undergraduate students' presses on left and right buttons occasionally made available points exchangeable for money. Blue lights over the buttons were correlated with multiple random-ratio random-interval components; usually, the random-ratio schedule was assigned to the left button and the random-interval to the right. During interruptions on the multiple schedule, students filled out sentence-completion guess sheets (e.g., The way to earn points with the left button is to...). For different groups, guesses were shaped with differential points also worth money (e.g., successive approximations to “press fast” for the left button), or were instructed (e.g., Write “press slowly” for the left button), or were simply collected. Control of rate of pressing by guesses was examined in individual cases by reversing shaped or instructed guesses, by instructing pressing rates, and/or by reversing multiple-schedule contingencies. Shaped guesses produced guess-consistent pressing even when guessed rates opposed those characteristic of the contingencies (e.g., slow random-ratio and fast random-interval rates), whereas guesses and rates of pressing rarely corresponded after unsuccessful shaping of guesses or when guessing had no differential consequences. Instructed guesses and pressing were inconsistently related. In other words, when verbal responses were shaped (contingency-governed), they controlled nonverbal responding. When they were instructed (rule-governed), their control of nonverbal responding was inconsistent: the verbal behavior sometimes controlled, sometimes was controlled by, and sometimes was independent of the nonverbal behavior.     

Effects of social context, reinforcer probability, and reinforcer magnitude on humans' choices to compete or not to compete.

In the first two experiments, subjects' choices to earn points (exchangeable for money) either by competing with a fictitious opponent or by not competing were studied. Buskist, Barry, Morgan, and Rossi's (1984) competitive fixed-interval schedule was modified to include a second response option, a noncompetitive fixed-interval schedule. After choosing to enter either option, the opportunity for reinforcers became available after the fixed-interval's duration had elapsed. Under the no-competition condition, points were always available after the interval had elapsed. Under the competition condition, points were available based on a predetermined probability of delivery. Experiments 1 and 2 examined how reinforcer probabilities and reinforcer magnitudes affected subjects' choices to compete. Several general conclusions can be made about the results: (a) Strong preferences to compete were observed at high and moderate reinforcer probabilities; (b) competing was observed even at very low reinforcer probabilities; (c) response rates were always higher in the competition component than in the no-competition component; and (d) response rates and choices to compete were insensitive to reinforcer-magnitude manipulations. In Experiment 3, the social context of this choice schedule was removed to determine whether the high levels of competing observed in the first two experiments were due to a response preference engendered by the social context provided by the experimenters through instructions. In contrast to the first two experiments, these subjects preferred the 60-s fixed-interval schedule (formerly the no-competition option), indicating that the instructions themselves were responsible for the preference to compete. This choice paradigm may be useful to future researchers interested in the effects of other independent variables (e.g., drugs, social context, instructions) on competitive behavior.     

Effects of alternative reinforcement on human behavior: the source does matter

Competing theories regarding the effects of delivering periodic response-independent reinforcement (more accurately, response-independent points exchanged for money) on a baseline rate of behavior were evaluated in human subjects. Contiguity theory holds that these events decrease target responding because incompatible behavior is adventitiously strengthened when the point deliveries follow target behavior closely in time. Matching theory holds that response-independent points, like any other alternative reinforcer, should reduce target responding. On this view, temporal contiguity between target responding and response-independent point delivery is unimportant. In our experiment, four different responses (moving a joystick in four different directions) were reinforced with points exchangeable for money according to four independent variable-interval schedules. Different schedules of point delivery were then superimposed on these baselines. When all superimposed point deliveries occurred immediately after one of the four responses (the target response), time allocated to target responding increased. When the superimposed point deliveries could be delivered at any time, time allocated to target responding declined and other behavior increased. When superimposed points could never immediately follow target responses, time allocated to target responding decreased further and other behavior or pausing predominated. The findings underscore the contribution of temporal contiguity in the effects of response-independent deliveries of food, money, points, etc.     

Effects of reinforcer consumption and magnitude on response rates during noncontingent reinforcement

Results of previous research on the effects of noncontingent reinforcement (NCR) have been inconsistent when magnitude of reinforcement was manipulated. We attempted to clarify the influence of NCR magnitude by including additional controls. In Study 1, we examined the effects of reinforcer consumption time by comparing the same magnitude of NCR when session time was and was not corrected to account for reinforcer consumption. Lower response rates were observed when session time was not corrected, indicating that reinforcer consumption can suppress response rates. In Study 2, we first selected varying reinforcer magnitudes (small, medium, and large) on the basis of corrected response rates observed during a contingent reinforcement condition and then compared the effects of these magnitudes during NCR. One participant exhibited lower response rates when large-magnitude reinforcers were delivered; the other ceased responding altogether even when small-magnitude reinforcers were delivered. We also compared the effects of the same NCR magnitude (medium) during 10-min and 30-min sessions. Lower response rates were observed during 30-min sessions, indicating that the number of reinforcers consumed across a session can have the same effect as the number consumed per reinforcer delivery. These findings indicate that, even when response rate is corrected to account for reinforcer consumption, larger magnitudes of NCR (defined on either a per-delivery or per-session basis) result in lower response rates than do smaller magnitudes.     

Earning and obtaining reinforcers under concurrent interval scheduling

Contingencies of reinforcement specify how reinforcers are earned and how they are obtained. Ratio contingencies specify the number of responses that earn a reinforcer, and the response satisfying the ratio requirement obtains the earned reinforcer. Simple interval schedules specify that a certain time earns a reinforcer, which is obtained by the first response after the interval. The earning of reinforcers has been overlooked, perhaps because simple schedules confound the rates of earning reinforcers with the rates of obtaining reinforcers. In concurrent variable-interval schedules, however, spending time at one alternative earns reinforcers not only at that alternative, but at the other alternative as well. Reinforcers earned for delivery at the other alternative are obtained after changing over. Thus the rates of earning reinforcers are not confounded with the rate of obtaining reinforcers, but the rates of earning reinforcers are the same at both alternatives, which masks their possibly differing effects on preference. Two experiments examined the separate effects of earning reinforcers and of obtaining reinforcers on preference by using concurrent interval schedules composed of two pairs of stay and switch schedules (MacDonall, 2000). In both experiments, the generalized matching law, which is based on rates of obtaining reinforcers, described responding only when rates of earning reinforcers were the same at each alternative. An equation that included both the ratio of the rates of obtaining reinforcers and the ratio of the rates of earning reinforcers described the results from all conditions from each experiment.     

Effects of reinforcer rate and reinforcer quality on time allocation: Extensions of matching theory to educational settings

We examined how 3 special education students allocated their responding across two concurrently available tasks associated with unequal rates and equal versus unequal qualities of reinforcement. The students completed math problems from two alternative sets on concurrent variable-interval (VI) 30-s VI 120-s schedules of reinforcement. During the equal-quality reinforcer condition, high-quality (nickels) and low-quality items ("program money" in the school's token economy) were alternated across sessions as the reinforcer for both sets of problems. During the unequal-quality reinforcer condition, the low-quality reinforcer was used for the set of problems on the VI 30-s schedule, and the high-quality reinforcer was used for the set of problems on the VI 120-s schedule. Equal- and unequal-quality reinforcer conditions were alternated using a reversal design. Results showed that sensitivity to the features of the VI reinforcement schedules developed only after the reinforcement intervals were signaled through countdown timers. Thereafter, when reinforcer quality was equal, the time allocated to concurrent response alternatives was approximately proportional to obtained reinforcement, as predicted by the matching law. However the matching relation was disrupted when, as occurs in most natural choice situations, the quality of the reinforcers differed across the response options.     

Humans' sensitivity to variation in reinforcer amount: Effects of the method of reinforcer delivery

Two experiments examined human subjects' sensitivity to variation in reinforcer amount under different methods of reinforcer delivery. Subjects chose between schedules varying in terms of amount and/or delay of reinforcement, the reinforcer being points exchangeable for money. In Experiment 1, reinforcer amount was manipulated by varying the monetary value of the points across conditions while the number of seconds of access to a consummatory response remained constant. Choice was strongly sensitive to reinforcer amount and indicative of self-control, as in previous experiments. In Experiment 2, reinforcer amount was manipulated by automatically delivering different numbers of points during the amount period, and the consummatory response was eliminated. Sensitivity to variation in reinforcer amount was significantly lower than in Experiment 1. Furthermore, the subjects in Experiment 2 exhibited significantly less self-control than did the subjects in Experiment 1. Humans' sensitivity to variation in reinforcer amount appears to be affected by factors that enhance the discrimi-nability of the consequences of responding.     

The effects of constant versus varied reinforcers on preference and resistance to change

Previous research has demonstrated that factors such as reinforcer frequency, amount, and delay have similar effects on resistance to change and preference. In the present study, 4 boys with autism made choices between a constant reinforcer (one that was the same food item every trial) and a varied food reinforcer (one that varied randomly between three possible food items). For all 4 boys, varied reinforcers were preferred over constant reinforcers, and they maintained higher response rates than constant reinforcers. In addition, when a distraction (a video clip) was introduced, responding maintained by varied reinforcers was more resistant to distraction than responding maintained by constant reinforcers. Thus, the present experiment extended the generality of the relation between preference and resistance to change to variation in reinforcer quality.     

Stimulus Equivalence: Effects Of A Default-response Option On Emergence Of Untrained Stimulus Relations

Human subjects were exposed to a concurrent-chains schedule in which reinforcer amounts, delays, or both were varied in the terminal links, and consummatory responses were required to receive points that were later exchangeable for money. Two independent variable-interval 30-s schedules were in effect during the initial links, and delay periods were defined by fixed-time schedules. In Experiment 1, subjects were exposed to three different pairs of reinforcer amounts and delays, and sensitivity to reinforcer amount and delay was determined based on the generalized matching law. The relative responding (choice) of most subjects was more sensitive to reinforcer amount than to reinforcer delay. In Experiment 2, subjects chose between immediate smaller reinforcers and delayed larger reinforcers in five conditions with and without timeout periods that followed a shorter delay, in which reinforcer amounts and delays were combined to make different predictions based on local reinforcement density (i.e., points per delay) or overall reinforcement density (i.e., points per total time). In most conditions, subjects' choices were qualitatively in accord with the predictions from the overall reinforcement density calculated by the ratio of reinforcer amount and total time. Therefore, the overall reinforcement density appears to influence the preference of humans in the present self-control choice situation.     

Within-session rates of responding when reinforcer magnitude is changed within the session

In the present experiment, the authors investigated the idea that within-session changes in operant response rates occur because subjects sensitize and then habituate to the reinforcer. If that is true, then altering an aspect of the reinforcer within the session should alter the observed within-session responding. The authors tested that idea by having rats press a lever for 2 food-pellet reinforcers delivered by a variable-interval 120-s schedule during 60-min baseline sessions. In treatment conditions, the magnitude of the reinforcer was halved (1 pellet) or doubled (4 pellets) 10, 20, 30, 40, or 50 min into the session. That magnitude of reinforcement then remained in effect for the rest of the session. Altering reinforcer magnitude altered the rates of responding within the session in a fashion consistent with the habituation explanation, that is, response rates increased, relative to baseline, when the magnitude of reinforcement was increased. They decreased when the magnitude was decreased. Those results were seemingly inconsistent with the competing idea that within-session decreases in responding rates are produced by satiation.     

Resistance to change and the law of effect

Three experiments using multiple schedules of reinforcement explored the implications of resistance-to-change findings for the response-reinforcer relation described by the law of effect, using both steady-state responding and responding recorded in the first few sessions of conditions. In Experiment 1, when response-independent reinforcement was increased during a third component, response rate in Components 1 and 2 decreased. This response-rate reduction was proportionately greater in a component in which reinforcer magnitude was small (2-s access to wheat) than in the component in which it was large (6-s access to wheat). However, when reinforcer rates in the two components were varied together in Experiments 2 and 3, response-rate change was the same regardless of the magnitude of reinforcers used in the two components, so that sensitivity of response rates to reinforcer rates (Experiment 2) and of response-rate ratios to reinforcer-rate ratios (Experiment 3) was unaffected by the magnitude of the reinforcers. Therefore, the principles determining resistance to change, described by behavioral momentum theory, seem not to apply when the source of behavior change is the variation of reinforcement contingencies that maintain the behavior. The use of extinction as a manipulation to study resistance to change is questioned.     

Induction when rats lick for 1% liquid-sucrose reinforcement

Previous research reported that rats responding for 1% liquid-sucrose reinforcement when 32% sucrose reinforcement is upcoming will decrease their response rate (contrast) if licking is the dependent measure and increase their response rate (induction) if pressing a lever is the dependent measure. The present study investigated whether induction could be observed when licking served as the dependent measure and whether induction in lever pressing and contrast in licking behaviour could be concurrently observed. Experiment 1 found induction when rats licked to earn the rewards but consumed them at a location separate from the spout licked to earn them. Experiment 2 also found induction when rats earned (and consumed) rewards by licking the same spout throughout the session. Experiment 3 separately measured instrumental lever pressing for sucrose rewards and licking the sucrose during the reward period. We found that both measures increased for 1% sucrose when 32% sucrose reinforcement was upcoming. The present results indicate that the type of response is not the sole determinant of whether contrast or induction is observed. Rather, they suggest that other procedural details, such as the location of reinforcer delivery, influence which effect is observed. The results also indicate that associative processes underlie the appearance of induction in responding for 1% sucrose.     

Induction when rats lick for 1% liquid-sucrose reinforcement

Previous research reported that rats responding for 1% liquid-sucrose reinforcement when 32% sucrose reinforcement is upcoming will decrease their response rate (contrast) if licking is the dependent measure and increase their response rate (induction) if pressing a lever is the dependent measure. The present study investigated whether induction could be observed when licking served as the dependent measure and whether induction in lever pressing and contrast in licking behaviour could be concurrently observed. Experiment 1 found induction when rats licked to earn the rewards but consumed them at a location separate from the spout licked to earn them. Experiment 2 also found induction when rats earned (and consumed) rewards by licking the same spout throughout the session. Experiment 3 separately measured instrumental lever pressing for sucrose rewards and licking the sucrose during the reward period. We found that both measures increased for 1% sucrose when 32% sucrose reinforcement was upcoming. The present results indicate that the type of response is not the sole determinant of whether contrast or induction is observed. Rather, they suggest that other procedural details, such as the location of reinforcer delivery, influence which effect is observed. The results also indicate that associative processes underlie the appearance of induction in responding for 1% sucrose.     

Behavior-dependent reinforcer-rate changes in concurrent schedules: A further analysis

Six pigeons were trained on concurrent variable-interval schedules in three different procedures. The first procedure was a standard concurrent schedule, and the relative reinforcer frequency for responding was varied. The second was a schedule in which a relative left-key response rate (over a fixed period of time) exceeding .75 produced, in the next identical time period a higher reinforcer rate on the right key. If this criterion was not exceeded, equal reinforcer rates were arranged on the two keys in this period. This was the dependent procedure. In the third (independent) procedure, the periods of higher right-key reinforcer rates occurred with the same probability as in the second procedure, but occurred independently of behavior. In the second and third procedures, the fixed-time period (window) was varied from 5 s to 60 s, and to 240 s in the second procedure only. Performance on the two keys was similar in the concurrent and independent procedures. The procedure used in the dependent conditions generally affected performance when the windows were shorter than about 30 s. Models of performance that assume that subjects do not discriminate changes in local relative reinforcer rates cannot account for the data. Moreover, existing models are inherently unable to account for the effects of contingencies of reinforcement between responding on one alternative and gaining reinforcers on another that are arranged or that emerge as a result of time allocated to alternative schedules. Undermatching on concurrent variable-interval schedules may result from such emergent contingencies.     

Reinforcer value may change within experimental sessions

Large and systematic changes in response rates often occur within sessions during operant conditioning procedures. In the present experiment, we asked whether the value of the reinforcer that supports responding also changes within sessions. Pigeons pecked a key for mixed grain available throughout the session. Occasionally, wheat was also provided for pecking a second key. The ratio of the rates of responding for mixed grain and wheat, a frequently used measure of relative reinforcer value, changed significantly within sessions when mixed grain was provided at high, but not at low, rates. Habituation to the reinforcer provides the most likely explanation for these changes in reinforcer value. Eventually, habituation may provide a unified explanation for the within-session changes in behavior that occur when many species of subjects respond on a wide variety of tasks.     

Running and responding reinforced by the opportunity to run: effect of reinforcer duration.

The present study investigated the effect of reinforcer duration on running and on responding reinforced by the opportunity to run. Eleven male Wistar rats responded on levers for the opportunity to run in a running wheel. Opportunities to run were programmed to occur on a tandem fixed-ratio 1 variable-interval 30-s reinforcement schedule. Reinforcer duration varied across conditions from 30 to 120 s. As reinforcer duration increased, the rates of running and lever pressing declined, and latency to lever press increased. The increase in latency to respond was consistent with findings that unconditioned inhibitory aftereffects of reinforcement increase with reinforcer magnitude. The decrease in local lever-pressing rates, however, was inconsistent with the view that response strength increases with the duration of the reinforcer. Response rate varied inversely, not directly, with reinforcer duration. Furthermore, within-session data challenge satiation, fatigue, and response deprivation as determinants of the observed changes in running and responding. In sum, the results point to the need for further research with nonappetitive forms of reinforcement.     

Interpersonal contingencies: Performance differences and cost-effectiveness

Three reinforcement contingencies were compared with regard to performance differences and cost-effectiveness (i.e., responses per unit reinforcer). Pairs of college students were studied under individual, cooperative, or competitive contingencies using a concurrent setting that included one of these three contingencies as one alternative and a lower paying individual contingency as the other alternative. With the individual and the cooperative contingencies, overall response rates were typically high; under competitive contingencies the overall response rates were substantially lower. Subjects responded at very high rates when competing, but chose not to compete most of the time. Competition and cooperation produced the most cost-effective responding, assessed as the number of responses made per $.01 of reinforcer. High overall rates of competitive responding were obtained when the contests were longer and the lower paying alternative contingency was not available.