EMERGENCE OF REINFORCER PREFERENCE AS A FUNCTION OF SCHEDULE REQUIREMENTS AND STIMULUS SIMILARITY

Tustin (1994) recently observed that an individual's preference for one of two concurrently available reinforcers under low schedule requirements (concurrent fixed-ratio [FR] 1) switched to the other reinforcer when the schedule requirements were high (concurrent FR 10). We extended this line of research by examining preference for similar and dissimilar reinforcers (i.e., those affecting the same sensory modality and those affecting different sensory modalities). Two individuals with developmental disabilities were exposed to an arrangement in which pressing two different panels produced two different reinforcers according to progressively increasing, concurrent-ratio schedules. When two dissimilar stimuli were concurrently available (food and a leisure item), no clear preference for one item over the other was observed, regardless of the FR schedules in effect (FR 1, 2, 5, 10, and 20). By contrast, when two similar stimuli were concurrently available (two food items), a clear preference for one item emerged as the schedule requirements were increased from FR 1 to FR 5 or FR 10. These results are discussed in terms of implications for conducting preference assessments and for selecting reinforcers to be used under training conditions in which response requirements are relatively high or effortful.     

ON THE RELATIVE REINFORCING EFFECTS OF CHOICE AND DIFFERENTIAL CONSEQUENCES

Research on the reinforcing effects of providing choice-making opportunities to individuals with developmental disabilities (i.e., allowing them to choose reinforcers or tasks) has produced inconsistent results, perhaps because the mechanisms underlying such effects remain unclear. Choice may produce a reinforcement effect because it is correlated with differential consequences (i.e., choice may increase one's access to higher preference stimuli), or it may have reinforcement value independent of (or in addition to) the chosen stimulus. In Experiment 1, we used a concurrent-operants arrangement to assess preference for a choice condition (in which participants selected one of two available reinforcers) relative to a no-choice condition (in which the therapist selected the same reinforcers on a yoked schedule). All 3 participants preferred the choice option. In Experiment 2, we altered the schedules so that the participant selected one of two lower preference reinforcers in the choice condition, whereas the therapist selected a higher preference stimulus for the participant either half or all of the time in the no-choice condition. Participants typically allowed the therapist to select reinforcers for them (i.e., they allocated responding to the no-choice condition) when it resulted in greater access to higher preference stimuli.     

A Reinforcer Pathology perspective on relapse

The Reinforcer Pathology theory proposes conditions under which drugs emerge as excessively preferred reinforcers compared to other available reinforcers among drug users. The theory highlights 2 key variables as important determinants of drug preference: (a) excessive preference for immediate rewards (high discounting of future rewards); and (b) excessive valuation for addictive reinforcers (e.g., drugs). Two iterations of the Reinforcer Pathology theory exist with the latest iteration (i.e., Reinforcer Pathology 2.0) specifying that the temporal window of reinforcer integration (measured by delay discounting) is a determinant of reinforcer value. The 2 iterations of the Reinforcer Pathology theory are described. A novel insight and understanding of abstinence and relapse from a Reinforcer Pathology perspective, limitations, and future directions are discussed. The Reinforcer Pathology theory continues the long-standing efforts to scientifically understand and better define novel concepts and methods to further translational research and improve treatment outcomes. Exploring the complementary relation between the Reinforcer Pathology perspective and other current approaches could have a valuable effect.     

Further evaluation of the use of preference categories to identify novel reinforcers: A systematic replication

Preference assessments are frequently used to identify reinforcers for individuals with autism spectrum disorder and intellectual and developmental disabilities. Although well‐established procedures have been shown to identify preferred stimuli for individual items that can be used in skill acquisition and behavior reduction programs, little research has been conducted on identifying categories of preferred items. In this study, paired‐stimulus preference assessments were conducted with 4 individuals with autism spectrum disorder. Researchers classified the edible stimuli as belonging to 1 of 4 categories: chocolate (e.g., chocolate chips and M&M's^®), salty/crunchy (e.g., chips and crackers), gummy (e.g., Swedish Fish^® and Starburst^®), or fruit/vegetable (e.g., grape and apple). Preference hierarchies were identified for individual stimuli and for categories of stimuli. For all participants, at least 3 of the 4 most preferred items came from the same category. Novel items (i.e., items not included in the preference assessments) identified by the researchers as belonging to the high‐preference category functioned as effective reinforcers during subsequent reinforcer assessments for all participants. This finding suggests that clinicians could identify likely effective edible reinforcers based on an individual's categorical preference without explicit testing.     

Some determinants of changes in preference over time

Results of longitudinal studies suggest that the stability of preferences varies across individuals, although it is unclear what variables account for these differences. We extended this work by conducting periodic assessments of preference for leisure activities over 3 to 6 months with 10 adults with developmental disabilities. Although previous research has collectively shown that preferences identified via repeated assessment are highly variable, our results showed that preferences were relatively stable for the majority (80%) of participants. In an attempt to identify some environmental determinants of shifts in preference, we provided extended daily access to high-preference items (preference-weakening manipulation) and paired access to low-preference items with social and edible putative reinforcers during brief sessions (preference-strengthening manipulation). Preference assessments continued over the course of these manipulations with 2 participants. Results showed that changes in preference across time could be produced systematically and suggest that naturally occurring changes in establishing operations or conditioning histories contribute to temporal shifts in preference. Implications for preference assessments, reinforcer usage, and planned attempts to change preferences are discussed.     

Preference for starting and finishing behavior patterns

Pigeon's key pecking was reinforced with food in two experiments in which the correspondence between preference for starting one of two reinforced behavior patterns and the likelihood of finishing it subsequently was examined. Reinforcers were scheduled according to concurrent schedules for two classes of interresponse times, modified such that reinforcers followed a center-key peck terminating either a shorter interresponse time started by a left-key peck or a longer interresponse time started by a right-key peck. In Experiment 1, the times when reinforcers potentially were available were not discriminated, whereas in Experiment 2 they were. Absolute reinforced pattern durations were varied. The relative frequency of starting a particular pattern was highly correlated with relative frequency of that completed pattern in both experiments. Other relations between starting and finishing a pattern depended on whether reinforced interresponse times were discriminated. For instance, preference for starting a pattern sometimes correlated negatively with the likelihood of subsequently completing it. The present experiments are described as capturing part of the ordinary language meaning of "intention," according to which an organism's behavior at one moment sets the occasion for an observer to say that the organism "intends" in the future to engage in one behavior rather than another.     

Pigeons' preference for free choice: number of keys versus key area.

In concurrent-chains schedules, pigeons prefer terminal links that provide two keys correlated with reinforcers (free choice) over those that provide only one key (forced choice), terminal-link reinforcement rates being equal. With same-size keys, free choice provides a larger area available for pecking. Preferences were examined using terminal links that differed in key number only (one or two) or key size only (small and medium or medium and large), or that equated the area of the two free-choice keys with that of the forced-choice key. Medium (standard) keys were typically preferred to small keys, but indifference was typically obtained between medium and large keys. The size preference usually overrode free-choice preference with one medium key pitted against two small keys, but free-choice preference was reliably observed with one large key pitted against two medium keys. In other words, preferences were a joint function of key number and key area, implying that free-choice preference is not reducible to preference for larger key areas. Free-choice preference requires separate keys rather than larger areas; the relevant behavioral units are the discriminated operants correlated with each terminal-link key rather than classes defined by topographical features such as area or perimeter.     

Developing and assessing stimulus control based on establishing operations during mand training using representative objects

The acquisition of verbal behavior can be difficult for some individuals with severe developmental disabilities. For two individuals who previously did not acquire picture exchange communication despite extensive efforts, we evaluated the effectiveness of using representative objects to teach mands for preferred items. In addition, we replicated and extended the findings of our previous research manipulating motivating operations (MO) for functional discriminated mands. Two individuals with developmental disabilities participated in a training procedure designed to teach two separate mands for two separate preferred items. Participants were taught to mand (e.g., for chips) using representative objects (e.g., empty bag of chips). Following training, the MO manipulation was used to assess for discriminated manding. This manipulation involved providing free access to one of the preferred items, such that there should be no motivation to ask for it, while motivation to ask for the other item remained in place. Results showed that both participants acquired at least a degree of discriminated manding using representative objects. At a minimum, a method was established to test complex discrimina‐ tions for preferred objects. Although we previously used this procedure in teaching picture card exchange discriminations, the current research extended previous research to individuals who were previously unresponsive to training and by using representative objects. Copyright © 2010 John Wiley & Sons, Ltd.     

A behavioral economic analysis of concurrently available money and cigarettes.

In economic terms, consumption of a reinforcer is determined by its price and the availability and price of other reinforcers. This study examined the effects of response-requirement (i.e., price) manipulations on the self-administration of two concurrently available reinforcers. Six cigarette smokers participated in 4-hr sessions in which money and puffs on a cigarette were concurrently available according to fixed-ratio schedules of reinforcement. Once stable responding was obtained with both reinforcers available at Fixed Ratio 100, the response requirement for one reinforcer was systematically varied (Fixed Ratio 1,000 and 2,500), while the other reinforcer remained scheduled at Fixed Ratio 100. Increasing the fixed-ratio size for a reinforcer decreased its consumption, with a greater decrease occurring for monetary reinforcement. This finding was quantified in economic terms as own-price elasticity, with elasticity coefficients greater for money than cigarettes. The effects of fixed-ratio size on response output also differed across the two reinforcers. Although greater responding occurred for money at Fixed Ratio 100, increases in fixed-ratio size (for money) decreased responding for money, whereas the same increase in fixed-ratio size (for puffs) increased responding for puffs. Finally, increasing the fixed-ratio size for one reinforcer had little effect on consumption of the other concurrently available reinforcer. This finding was quantified as cross-price elasticity, with elasticity coefficients near 0.0 for most subjects, indicating little or no reinforcer interaction. The results indicate that the reinforcing effects of cigarettes and money in the setting studied here differed, and that the effects produced by changing the price of one reinforcer did not interact with the consumption of the other concurrently available reinforcer.     

AN ANALYSIS OF REINFORCER SUBSTITUTABILITY USING OBJECT MANIPULATION AND SELF-INJURY AS COMPETING RESPONSES

The concept of reinforcer substitutability proposes a continuum of interactions among reinforcers in a given situation. At one end of this continuum, reinforcers are substitutable, with one reinforcer being readily traded for another. We conducted an analysis of reinforcers that were substitutable with those produced by self-injurious behavior (SIB). Three individuals with profound developmental disabilities, whose SIB appeared to be maintained by automatic reinforcement, participated. Results of three experiments showed that (a) object manipulation and SIB were inversely related when leisure materials and SIB were concurrently available, with participants showing almost complete preference for object manipulation; (b) attempts to reduce SIB using the preferred objects as reinforcers in differential reinforcement contingencies were unsuccessful for all 3 participants; and (c) participants' preferences for SIB or object manipulation systematically changed when reinforcer cost (the amount of effort required to obtain the object) was varied. Results of the three experiments illustrate the importance of examining interactions among concurrently available reinforcers when conducting reinforcer assessments.     

Choice between delayed reinforcers in a discrete-trials schedule: The effect of deprivation level

Choice between two reinforcers differing in magnitude and delay was investigated in rats using a discrete-trials schedule in which the two reinforcers were associated with two levers (A and B); in each session 5 free-choice trials (A and B both available) were interspersed among 44 forced-choice trials (A alone, 22 trials; B alone, 22 trials). In Experiment 1, preference for the more concentrated of two sucrose solutions declined as the delay to that reinforcer was progressively increased. In Experiment 2, progressively increasing the delay to both reinforcers by the same amount resulted in a shift in preference away from the less concentrated solution. In Experiment 3, it was found that the decline in preference for the more concentrated solution as a function of the delay to that reinforcer was steeper when the rats were maintained at 90% than when they were maintained at 80% of their free-feeding body weights. This effect of deprivation level on choice is inconsistent with some current models of “self-control”.     

The effects of constant versus varied reinforcers on preference and resistance to change

Previous research has demonstrated that factors such as reinforcer frequency, amount, and delay have similar effects on resistance to change and preference. In the present study, 4 boys with autism made choices between a constant reinforcer (one that was the same food item every trial) and a varied food reinforcer (one that varied randomly between three possible food items). For all 4 boys, varied reinforcers were preferred over constant reinforcers, and they maintained higher response rates than constant reinforcers. In addition, when a distraction (a video clip) was introduced, responding maintained by varied reinforcers was more resistant to distraction than responding maintained by constant reinforcers. Thus, the present experiment extended the generality of the relation between preference and resistance to change to variation in reinforcer quality.     

CONSUMPTION AND RESPONSE OUTPUT AS A FUNCTION OF UNIT PRICE: MANIPULATION OF COST AND BENEFIT COMPONENTS

We conducted preference assessments with 4 typically developing children to identify potential reinforcers and assessed the reinforcing efficacy of those stimuli. Next, we tested two predictions of economic theory: that overall consumption (reinforcers obtained) would decrease as the unit price (response requirement per reinforcer) increased and that the cost and benefit components that defined unit price would not influence overall consumption considerably when unit price values were equal. We tested these predictions by arranging unit price such that the denominator was one (e.g., two responses produced one reinforcer) or two (e.g., four responses produced two reinforcers). Results showed that consumption decreased as unit price increased and that unit price values with different components produced similar consumption.     

Impulsive choice and pre‐exposure to delays: iv. effects of delay‐ and immediacy‐exposure training relative to maturational changes in impulsivity

Impulsive choice describes preference for smaller, sooner rewards over larger, later rewards. Excessive delay discounting (i.e., rapid devaluation of delayed rewards) underlies some impulsive choices, and is observed in many maladaptive behaviors (e.g., substance abuse, gambling). Interventions designed to reduce delay discounting may provide therapeutic gains. One such intervention provides rats with extended training with delayed reinforcers. When compared to a group given extended training with immediate reinforcers, delay‐exposed rats make significantly fewer impulsive choices. To what extent is this difference due to delay‐exposure training shifting preference toward self‐control or immediacy‐exposure training (the putative control group) shifting preference toward impulsivity? The current study compared the effects of delay‐ and immediacy‐exposure training to a no‐training control group and evaluated within‐subject changes in impulsive choice across 51 male Wistar rats. Delay‐exposed rats made significantly fewer impulsive choices than immediacy‐exposed and control rats. Between‐group differences in impulsive choice were not observed in the latter two groups. While delay‐exposed rats showed large, significant pre‐ to posttraining reductions in impulsive choice, immediacy‐exposed and control rats showed small reductions in impulsive choice. These results suggest that extended training with delayed reinforcers reduces impulsive choice, and that extended training with immediate reinforcers does not increase impulsive choice.     

Development and maintenance of choice in a dynamic environment

Four pigeons were exposed to a concurrent procedure similar to that used by Davison, Baum, and colleagues (e.g., Davison & Baum, 2000, 2006) in which seven components were arranged in a mixed schedule, and each programmed a different left∶right reinforcer ratio (1∶27, 1∶9, 1∶3, 1∶1, 3∶1, 9∶1, 27∶1). Components within each session were presented randomly, lasted for 10 reinforcers each, and were separated by 10-s blackouts. These conditions were in effect for 100 sessions. When data were aggregated over Sessions 16-50, the present results were similar to those reported by Davison, Baum, and colleagues: (a) preference adjusted rapidly (i.e., sensitivity to reinforcement increased) within components; (b) preference for a given alternative increased with successive reinforcers delivered via that alternative (continuations), but was substantially attenuated following a reinforcer on the other alternative (a discontinuation); and (c) food deliveries produced preference pulses (immediate, local, increases in preference for the just-reinforced alternative). The same analyses were conducted across 10-session blocks for Sessions 1-100. In general, the basic structure of choice revealed by analyses of data from Sessions 16-50 was preserved at a smaller level of aggregation (10 sessions), and it developed rapidly (within the first 10 sessions). Some characteristics of choice, however, changed systematically across sessions. For example, effects of successive reinforcers within a component tended to increase across sessions, as did the magnitude and length of the preference pulses. Thus, models of choice under these conditions may need to take into account variations in behavior allocation that are not captured completely when data are aggregated over large numbers of sessions.     

Effects of social context, reinforcer probability, and reinforcer magnitude on humans' choices to compete or not to compete.

In the first two experiments, subjects' choices to earn points (exchangeable for money) either by competing with a fictitious opponent or by not competing were studied. Buskist, Barry, Morgan, and Rossi's (1984) competitive fixed-interval schedule was modified to include a second response option, a noncompetitive fixed-interval schedule. After choosing to enter either option, the opportunity for reinforcers became available after the fixed-interval's duration had elapsed. Under the no-competition condition, points were always available after the interval had elapsed. Under the competition condition, points were available based on a predetermined probability of delivery. Experiments 1 and 2 examined how reinforcer probabilities and reinforcer magnitudes affected subjects' choices to compete. Several general conclusions can be made about the results: (a) Strong preferences to compete were observed at high and moderate reinforcer probabilities; (b) competing was observed even at very low reinforcer probabilities; (c) response rates were always higher in the competition component than in the no-competition component; and (d) response rates and choices to compete were insensitive to reinforcer-magnitude manipulations. In Experiment 3, the social context of this choice schedule was removed to determine whether the high levels of competing observed in the first two experiments were due to a response preference engendered by the social context provided by the experimenters through instructions. In contrast to the first two experiments, these subjects preferred the 60-s fixed-interval schedule (formerly the no-competition option), indicating that the instructions themselves were responsible for the preference to compete. This choice paradigm may be useful to future researchers interested in the effects of other independent variables (e.g., drugs, social context, instructions) on competitive behavior.     

Preference for fluent versus disfluent work schedules

Two studies were conducted that examined the preference of a student diagnosed with a brain injury. In Study 1, a preference assessment was followed by a three-choice concurrent-operants reinforcer assessment. Two choices resulted in access to preferred activities for completing work, and a third choice resulted in access to nothing (i.e., no activity). Unpredictably, the participant consistently chose the no-activity option. Study 2 examined why this student preferred work associated with no activity over preferred activities. Through a variety of concurrent-operants procedures, it was determined that she preferred fluent work followed by reinforcers rather than work that was broken up by access to preferred activities. Implications for research on preference are discussed.     

Further examination of factors that influence preference for positive versus negative reinforcement

Factors that influence choice between qualitatively different reinforcers (e.g., a food item or a break from work) are important to consider when arranging treatments for problem behavior. Previous findings indicate that children who engage in problem behavior maintained by escape from demands may choose a food item over the functional reinforcer during treatment (DeLeon, Neidert, Anders, & Rodriguez-Catter, 2001; Lalli et al., 1999). However, a number of variables may influence choice between concurrently available forms of reinforcement. An analogue for treatment situations in which positive reinforcement for compliance is in direct competition with negative reinforcement for problem behavior was used in the current study to evaluate several variables that may influence choice. Participants were 5 children who had been diagnosed with developmental disabilities and who engaged in problem behavior maintained by escape from demands. In the first phase, the effects of task preference and schedule of reinforcement on choice between a 30-s break and a high-preference food item were evaluated. The food item was preferred over the break, regardless of the preference level of the task or the reinforcement schedule, for all but 1 participant. In the second phase, the quality of the break was manipulated by combining escape with toys, attention, or both. Only 1 participant showed preference for the enriched break. In the third phase, choice of a medium- or low-preference food item versus the enriched break was evaluated. Three of 4 participants showed preference for the break over the less preferred food item. Results extend previous research by identifying some of the conditions under which individuals who engage in escape-maintained behavior will prefer a food reinforcer over the functional one.     

Choice: A local analysis

Analyses of free-operant choice usually employ one of two general procedures: the simple concurrent procedure (i.e., a concurrent variable-interval variable-interval schedule) or the concurrent-chains procedure (i.e., concurrently available initial links, each leading to an exclusively available terminal link). Theories about choice usually focus on only one of the two procedures. For example, maximization theories, which assert that behavior is distributed between two alternatives in such a way that overall rate of reinforcement is maximized, have been applied only to the simple concurrent procedure. In the present paper, a form of the pairing hypothesis (according to which pairings between one stimulus and another affect the value of the first, and pairings between responses and reinforcers affect the value of the former) is developed in a way that allows it to make qualitative predictions with regard to choice in a variety of simple concurrent and concurrent-chains procedures. The predictions include matching on concurrent variable-interval variable-interval schedules, preference reversal in the self-control paradigm, and preference for tandem over chained terminal links.     

Preference for reinforcers under varying schedule arrangements: A behavioral economic analysis

The field of behavioral economics combines concepts from economics and operant conditioning to examine the influence of schedules or price on preference for reinforcers. Three case studies are reported in which behavioral economic analyses were used to assess relative preference for reinforcers shown by people with intellectual disabilities when schedule requirements varied. The studies examined (a) preference for different reinforcers, (b) substitutability of reinforcers, and (c) changes in preference as a function of schedule requirements.