Choice between reliable and unreliable reinforcement alternatives revisited: Preference for unreliable reinforcement

Pigeons' choices between a reliable alternative that always provided food after a delay (i.e., 100% reinforcement) and an unreliable one that provided food or blackout equally often after a delay (i.e., 50% reinforcement) was studied using a discrete-trials concurrent-chains procedure modified to prevent choice between alternatives following a blackout outcome. Initial links were fixed-ratio 1 schedules, and terminal links were fixed-time schedules. Stimuli presented during the terminal-link delays were correlated with the food and blackout outcomes. In Experiment 1, terminal-link durations were varied. With short terminal links (i.e., 10 s), 6 of 8 subjects showed strong preference for the 50% side. As terminal-link duration increased to 30 s, preference, regardless of direction, became less extreme. In Experiment 2, the side-key location of the 50% and 100% alternatives was reversed for 3 subjects. Preference for the 50% alternative reoccurred following the key reversal. When a 5-s separation was subsequently interposed between the initial and terminal links for both alternatives, all birds reversed to a preference for the 100% side. In general, the strong preference for the 50% side was qualitatively consistent with the expectation that the procedure enhanced the conditioned-reinforcement effectiveness of the food-associated terminal-link stimulus on the 50% side. Implications of the results for various accounts of choice of the 50% alternative are discussed.     

An eight-alternative concurrent schedule: foraging in a radial maze.

In two experiments conducted in an eight-arm radial maze, food pellets were delivered when a photocell beam was broken at the end of each arm via a nose poke, according to either fixed-interval or random-interval schedules of reinforcement, with each arm providing a different frequency of reinforcement. The behavior of rats exposed to these procedures was well described by the generalized matching law; that is, the relationships between log behavior ratios and log pellet ratios were approximated by linear functions. The slopes of these log-log functions, an index of sensitivity to reinforcement frequency, were greatest for nose pokes, intermediate for time spent in an arm, and least for arm entries. Similar results were obtained with both fixed-interval and random-interval schedules. Addition of a 10-s changeover delay in both experiments eliminated the slope differentials between nose pokes and time spent in an arm by reducing the slopes of the nose-poke functions. These results suggest that different aspects of foraging may be differentially sensitive to reinforcement frequency. With concurrent fixed-interval schedules, the degree of temporal control exerted by individual fixed-interval schedules was directly related to reinforcement frequency.     

On the relation between preference and resistance to change

Nevin (1979) noted that preference in concurrent chains and resistance to change in multiple schedules were correlated, in that both measures were affected similarly by variations in parameters of reinforcement such as rate, immediacy, and magnitude. To investigate the relationship between preference and resistance to change directly, we used a within-session procedure that arranged concurrent chains in one half of the session and a multiple schedule in the other half. The same variable-interval schedules served as terminal links in concurrent chains and as the components of the multiple schedule, and were signaled by the same stimuli. After performances had stabilized, responding in the multiple schedule was disrupted by delivering response-independent reinforcement during the blackout periods between components. Both preference in concurrent chains and relative resistance to change of multiple-schedule responding were well described as power functions of relative reinforcement rate, as predicted by current quantitative models (Grace, 1994; Nevin, 1992b). In addition, unsystematic variation in preference and resistance to change was positively correlated, which suggests that preference and resistance to change are independent measures of a single construct. That construct could be described as the learning that occurs regarding the prevailing conditions of reinforcement in a distinctive stimulus situation.     

The control of choice by its consequences

Five pigeons were trained on concurrent variable-interval schedules in which equal rates of reinforcement were always arranged for left- and right-key responses, but different overall rates were signaled by key colors. Sessions began with both keys lit yellow for the instrumental phase. If, after 20 s of this phase, the relative number of responses that had been made to the left key equaled or exceeded .75, both keys changed red for the contingent phase. The contingent phase arranged another concurrent variable-interval schedule for a further 20 s before the instrumental phase was reinstated. However, if preference in the instrumental phase did not exceed .75, the instrumental phase continued for a further 20 s before preference was again compared with the criterion. In Part 1, the reinforcer rate arranged in the instrumental phase was held constant at 4.8 reinforcers per minute, while the reinforcer rate arranged in the contingent phase was varied across conditions from 0 to 19.2 over five steps. In Part 2, reinforcer rates in the contingent phase were kept constant at 36 per minute, while reinforcer rates in the instrumental phase were varied from 0 to 36 over seven steps. Part 3 replicated Part 2 but used reinforcer rates in both phases that were one third of those arranged in Part 2. Measures of choice obtained by summing responses across presentations of the instrumental phase became more extreme toward the left key as the reinforcer rate obtained in the contingent phase was increased (Part 1) and as the reinforcer rate obtained in the instrumental phase was decreased (Parts 2 and 3). Changes in these measures of choice were accompanied by systematic changes in the relative frequency with which the criterion was exceeded. Changes in both these measures were correlated with changes in the relative frequency with which subjects responded exclusively to the left key. These results are discussed with respect to the two choices that were concurrently available in this procedure and the response alternatives that might constitute the concurrent operants in each choice.     

BASIC AND APPLIED RESEARCH ON CHOICE RESPONDING

Choice responding refers to the manner in which individuals allocate their time or responding among available response options. In this article, we first review basic investigations that have identified and examined variables that influence choice responding, such as response effort and reinforcement rate, immediacy, and quality. We then describe recent bridge and applied studies that illustrate how the results of basic research on choice responding can help to account for human behavior in natural environments and improve clinical assessments and interventions.     

ON THE RELATIVE REINFORCING EFFECTS OF CHOICE AND DIFFERENTIAL CONSEQUENCES

Research on the reinforcing effects of providing choice-making opportunities to individuals with developmental disabilities (i.e., allowing them to choose reinforcers or tasks) has produced inconsistent results, perhaps because the mechanisms underlying such effects remain unclear. Choice may produce a reinforcement effect because it is correlated with differential consequences (i.e., choice may increase one's access to higher preference stimuli), or it may have reinforcement value independent of (or in addition to) the chosen stimulus. In Experiment 1, we used a concurrent-operants arrangement to assess preference for a choice condition (in which participants selected one of two available reinforcers) relative to a no-choice condition (in which the therapist selected the same reinforcers on a yoked schedule). All 3 participants preferred the choice option. In Experiment 2, we altered the schedules so that the participant selected one of two lower preference reinforcers in the choice condition, whereas the therapist selected a higher preference stimulus for the participant either half or all of the time in the no-choice condition. Participants typically allowed the therapist to select reinforcers for them (i.e., they allocated responding to the no-choice condition) when it resulted in greater access to higher preference stimuli.     

ASSESSMENT OF PREFERENCE FOR VARIED VERSUS CONSTANT REINFORCERS

One method that has been demonstrated to improve the effectiveness of reinforcement is stimulus (reinforcer) variation (Egel, 1980). Egel found that bar pressing increased and responding occurred more rapidly during varied reinforcement than during constant reinforcement when identical stimuli were used across phases for 10 individuals with autism. The purpose of the current investigation was to assess the preferences of 7 individuals for varied presentation of slightly lower quality stimuli relative to constant access to the highest quality stimulus. Varied presentation was preferred over constant reinforcer presentation with 4 participants, and the opposite was true for 2 participants. One participant did not demonstrate a preference. These results suggest that stimulus variation may allow less preferred reinforcers to compete effectively with a more highly preferred reinforcer for some individuals.     

EVALUATION OF CLIENT PREFERENCE FOR FUNCTION-BASED TREATMENT PACKAGES

Functional communication training (FCT) and noncontingent reinforcement (NCR) are commonly prescribed treatments that are based on the results of a functional analysis. Both treatments involve delivery of the reinforcer that is responsible for the maintenance of destructive behavior. One major difference between the two treatment procedures is that client responding determines reinforcement delivery with FCT (e.g., reinforcement of communication is delivered on a fixed-ratio 1 schedule) but not with NCR (e.g., reinforcement is delivered on a fixed-time 30-s schedule). In the current investigation, FCT and NCR were equally effective in reducing 2 participants' destructive behavior that was sensitive to attention as reinforcement. After the treatment analysis, the participants' relative preference for each treatment was evaluated using a modified concurrent-chains procedure. Both participants demonstrated a preference for the FCT procedure. The results are discussed in terms of treatment efficacy and preference for control over when reinforcement is delivered. In addition, a method is demonstrated in which clients with developmental disabilities can participate in selecting treatments that are designed to reduce their destructive behavior.     

Concurrent variable-interval variable-ratio schedules can provide only weak evidence for matching

Herrnstein and Heyman (1979) showed that when pigeons' pecking is reinforced on concurrent variable-interval variable-ratio schedules, (1) their behavior ratios match the ratio of the schedules' reinforcer frequencies, and (2) there is more responding on the variable interval. Since maximizing the reinforcement rate would require responding more on the variable ratio, these results were presented as establishing the primacy of matching over maximizing. In the present report, different ratios of behavior were simulated on a computer to see how they would affect reinforcement rates on these concurrent schedules. Over a wide range of experimenter-specified choice ratios, matching obtained — a result suggesting that changes in choice allocation produced changes in reinforcer frequencies that correspond to the matching outcome. Matching also occurred at arbitrarily selected choice ratios when reinforcement rates were algebraically determined by each schedule's reinforcement-feedback function. Additionally, three birds were exposed to concurrent variable-interval variable-ratio schedules contingent on key pecking in which hopper durations were varied in some conditions to produce experimenter-specified choice ratios. Matching generally obtained between choice ratios and reinforcer-frequency ratios at these different choice ratios. By suggesting that reinforcer frequencies track choice on this procedure, instead of vice versa, this outcome questions whether matching-as-outcome was due to matching-as-process in the Herrnstein and Heyman study.     

Melloration and maximization of reinforcement minus costs of behavior

Eight pigeons were exposed to independent concurrent schedules. Concurrent variable-interval 60-second variable-interval 60-second schedules were presented to one group of four subjects. Following baseline training, a limited hold was added to one of the schedules and the duration of the hold was decreased in successive conditions. Concurrent variable-interval 120-second variable-interval 40-second schedules were presented to another group of four subjects. These subjects were first exposed to decreasing durations of a limited hold in the variable-interval 40-second component. After replication of the baseline, a limited hold in the variable-interval 120-second component was decreased in duration. The initial durations of the holds were determined from the subjects' responding in the baseline conditions. A duration was chosen such that approximately 25% of the scheduled reinforcers would be canceled if responding remained unchanged.

Approximate matching of time proportions and reinforcement proportions was observed when the limited hold was added to the variable-interval 60-second schedule and when the limited hold was added to the variable-interval 40-second schedule. Time proportions were less extreme than reinforcement proportions when the limited hold operated in a variable-interval 120-second schedule. Overall reinforcement rates tended to decrease with continued training in concurrent schedules with a limited hold. Absolute deviations from time matching also decreased. The results provide evidence against the principle of reinforcement maximization, and support Herrnstein and Vaughan's (1980) melioration hypothesis.