Behavioral economics and within-session changes in responding

Pigeons and rats responded on fixed-ratio schedules with requirements ranging from 5 to 120 responses. Consistent with past results from several schedules and procedures, responding usually changed systematically within experimental sessions. The within-session changes were usually larger and were less symmetrical around the middle of the session for schedules that provided higher, rather than lower, rates of reinforcement. These results suggest that similar variables contribute to within-session changes in responding under different schedules. When an economic demand function was fit to the data, the intensity and elasticity of demand for food and the percentage of the variance accounted for decreased within sessions, although the trend for elasticity did not reach statistical significance for pigeons. These results suggest that relatively short sessions should be used to study economic demand in open economies and that demand may differ at different times in a session and in sessions of different lengths. Within-session changes in intensity, but not necessarily elasticity, of demand are consistent with behavioral economic theories.     

Economic concepts for the analysis of behavior

A review of the relationship between schedule of reinforcement, response rate, and choice suggests that certain unifying concepts from economics can contribute to a more complete science of behavior. Four points are made: 1) a behavioral experiment is an economic system and its characteristics—open or closed—can strongly determine the results; 2) reinforcers can be distinguished by a functional property called elasticity; 3) reinforcers may interact as complements as well as substitutes; 4) no simple choice rule, such as strict matching, can account for all choice behavior.     

Behavioral economics and behavioral momentum

Some relations between elasticity of demand and the conditions of reinforcement are reanalyzed in terms of resistance to change, in ways suggested by the metaphor of behavioral momentum; some relations between resistance to change and the conditions of reinforcement are reanalyzed in terms of elasticity of demand, in ways suggested by behavioral economics. In addition, some data on labor supply in relation to variable-ratio schedules and alternative reinforcement are reanalyzed in terms of resistance to change and compared with steady-state resistance data for performance on multiple and concurrent interval schedules. The results of these studies can be summarized by two functions based on the behavioral momentum approach, relating relative behavioral mass to relative reinforcement per response or per unit time. The former is a relation between relative unit price and relative behavioral mass, suggesting the possibility of convergent measurement of a theoretical construct common to both approaches. However, the momentum and economic approaches differ fundamentally on whether it is preferable to construe discriminated operant behavior as selected and strengthened by its consequences or as part of a behavior–consequence bundle that maximizes utility.     

Reward Density And Variable-interval Schedule Performance In An Open Economy

There is no general agreement regarding the form of the relation between response rate and reinforcement rate when single schedules of reinforcement are studied in an open economy. The present study assessed the form of this relation using reward density, which incorporates both reinforcement rate and duration of access to food, as an independent variable. Reward density was manipulated with 4 pigeons by changing the value of the variable-interval schedule, the hopper duration, or both. The relations between response rate and reward density were sharply rising and hyperbolic in 3 of 4 pigeons, replicating results obtained by Catania and Reynolds (1968). Because eating efficiency was lower in conditions that provided longer hopper durations, programmed reward densities differed from obtained reward densities. When response rates were examined as a function of obtained reward densities, the same relations were demonstrated more strongly. In further clarifying the relation between response rate and reward density in an open economy, these results lend support to the conclusion that open and closed economies yield different behavioral effects.     

The magnitude-of-reinforcement function in closed and open economies.

It has been hypothesized that the magnitude-of-reinforcement effect may differ in closed and open experimental economies. We determined the relationship between magnitude of reinforcement and response rate in three feeding conditions: a closed economy in which total intake was unrestricted, a closed economy in which total intake was restricted so as to maintain body weight at 85% of free-feeding weight, and a traditional open economy in which subjects received food outside the experimental session. In the closed economies, regardless of body weight, the rats responded faster for smaller pellets and when the fixed ratio for pellets was higher. In the open economy, there was no reliable effect of pellet size or pellet cost on response rate. It is concluded that although there are circumstances in which response rate is an immediate function of the parameters of reinforcement, rate is not necessarily a measure of response strength. Response rate may instead, or additionally, contribute to a strategy of reducing the costs associated with resource utilization.     

A cluster analysis of text message users based on their demand for text messaging: A behavioral economic approach

The goal of this study was to determine whether cluster analysis could be used to identify distinct subgroups of text message users based on behavioral economic indices of demand for text messaging. Cluster analysis is an analytic technique that attempts to categorize cases based on similarities across selected variables. Participants completed a questionnaire about mobile phone usage and a hypothetical texting demand task in which they indicated their likelihood of paying an extra charge to continue to send text messages. A hierarchical cluster analysis was conducted on behavioral economic indices, such as demand intensity, demand elasticity, breakpoint, and the maximum expenditure. With the cluster analysis, we identified 3 subgroups of text message users. The groups were characterized by (a) high intensity and low elasticity, (b) high intensity and medium elasticity, and (c) low intensity and high elasticity. In a demonstration of convergent validity, there were statistically significant and conceptually meaningful differences across the subgroups in various measures of mobile phone use and text messaging. Cluster analysis is a useful tool for identifying and profiling distinct, practically meaningful groups based on behavioral indices and could provide a framework for targeting interventions more efficiently.     

Effects of economy type and nicotine on the essential value of food in rats

The exponential demand equation proposed by Hursh and Silberberg (2008) provides an estimate of the essential value of a good as a function of price. The model predicts that essential value should remain constant across changes in the magnitude of a reinforcer, but may change as a function of motivational operations. In Experiment 1, rats' demand for food across a sequence of fixed-ratio schedules was assessed during open and closed economy conditions and across one- and two-pellet per reinforcer delivery conditions. The exponential equation was fitted to the relation between fixed-ratio size and the logarithm of the absolute number of reinforcers. Estimates of the rate of change in elasticity of food, the proposed measure of essential value, were compared across conditions. Essential value was equivalent across magnitudes during the closed economy, but showed a slight decrease across magnitudes during the open economy. Experiment 2 explored the behavioral mechanisms of nicotine's effects on consumption with the results from Experiment 1 serving as a within-subject frame of reference. The same subjects were administered nicotine via subcutaneously implanted osmotic minipumps at a dose of 3 mg/kg/day and exposed to both the one- and two-pellet conditions under a closed economy. Although nicotine produced large decreases in demand, essential value was not significantly changed. The data from the present experiments provide further evidence for the adequacy of the exponential demand equation as a tool for quantifying the rate of change in elasticity of a good and for assessing behavioral mechanisms of drug action.     

Variable-interval schedule performance in open and closed economies

In two experiments, pigeons obtained food according to variable-interval schedules. In the first experiment, equivalent variable-interval schedules with average interreinforcer intervals ranging between 10 and 80 s in different conditions were studied in both open and closed economies. Response rates increased as reinforcement frequency decreased in the closed economy. By contrast, in the open economy response rates decreased for 1 bird and were variable for the other as reinforcement frequency decreased. The second experiment showed that the differences in the functions between responding and reinforcement frequency in the two types of economies were not due to changes in deprivation level. These results suggest that open and closed economies yield different behavioral effects. This conclusion is supported further by a reconsideration of previous findings that appear counter to the conclusion.     

Delayed reinforcement in a multiple schedule

Three rats and a pigeon were first trained on a two-component multiple schedule in which reinforcement in the two components occurred immediately after a response. Later, reinforcement in one component was delayed by a few seconds. During both stages of the experiment, reinforcement was scheduled by equal variable- (pigeon) or random-interval (rats) schedules in the two components. The main effect of the delayed reinforcement was to increase the rate of responding in the unchanged (non-delay) component. This behavioral contrast effect did not appear in all cases to be dependent upon a reduction in the rate of responding or the frequency of reinforcement in the delay component. This finding suggests that a reduction in response rate and/or reinforcement frequency in one component of a multiple schedule may not be a necessary prerequisite for the occurrence of behavioral contrast. This finding is, however, consistent with an explanation that suggests that behavioral contrast results from the introduction of a less-preferred condition in one component of a multiple schedule, since it is known that animals “prefer” immediate to delayed reinforcement.     

SKINNER AS MISSIONARY AND PROPHET: A REVIEW OF BURRHUS F. SKINNER: SHAPER OF BEHAVIOUR

Do we need another book about B. F. Skinner? According to Frederick Toates, the answer is “yes” because “there is still much to be said” (p. vii) about Skinner. In his recent biography, Burrhus F. Skinner: Shaper of Behaviour (2009), Toates attempts to integrate Skinner into the mainstream of psychology by showing areas of commonality between Skinner''s radical behaviorism and subdisciplines within psychology such as cognitive, social, and biological psychology. Admirably, although in some instances understandably naively, Toates attempts to demonstrate the power of positive reinforcement to explain myriad complex behaviors, including a fairly lengthy interpretation of religious behavior. In addition, Toates credits Skinner for being ahead of his time on both social and environmental issues. Toates falters, however, in his insistence that behavior analysis still needs and can benefit from cognitive concepts. He nevertheless provides an otherwise objective and sympathetic view of Skinner the person and the behavioral science he helped to create in a book that should be informative for both behavior analysts and those outside the field.     

Choice, time allocation, and response rate during stimulus generalization

Six pigeons were trained to discriminate between two noise intensities using a procedure that assessed choice, time allocation, and response rate simultaneously and independently. Responses on the left or right key (R1 or R2) were respectively correct in the presence of two different intensities, S1 and S2. After a correct response, reinforcement became available for pecks on the center key. Reinforcement density for R1¦S1 relative to R2¦S2 was varied across experimental conditions. Generalization tests followed extensive training at each condition. As a function of stimulus intensity, proportions of initial choices of R2, of time spent in R2-initiated components, and of center-key responses emitted in R2-initiated components all yielded sigmoidal gradients of similar slope, which shifted slightly in location when relative reinforcement density changed. Changeovers were maximal where initial choice proportions approximated 0.5. Gradients relating the absolute number of center-key responses to stimulus intensity were also roughly sigmoidal, but were more sensitive to changes in reinforcement density. Gradients of momentary response rate also depended on reinforcement density. During training, large but transitory shifts in choice responding occurred when reinforcement density changed, while differences in momentary response rate developed slowly, suggesting separate control of choice and response rate by the contingencies of reinforcement.     

Analytical Problems and Suggestions in the Analysis of Behavioral Economic Demand Curves

Behavioral economic demand curves (Hursh, Raslear, Shurtleff, Bauman, & Simmons, 1988) are innovative approaches to characterize the relationships between consumption of a substance and its price. In this article, we investigate common analytical issues in the use of behavioral economic demand curves, which can cause inconsistent interpretations of demand curves, and then we provide methodological suggestions to address those analytical issues.

We first demonstrate that log transformation with different added values for handling zeros changes model parameter estimates dramatically. Second, demand curves are often analyzed using an overparameterized model that results in an inefficient use of the available data and a lack of assessment of the variability among individuals.

To address these issues, we apply a nonlinear mixed effects model based on multivariate error structures that has not been used previously to analyze behavioral economic demand curves in the literature. We also propose analytical formulas for the relevant standard errors of derived values such as P _max, O _max, and elasticity. The proposed model stabilizes the derived values regardless of using different added increments and provides substantially smaller standard errors. We illustrate the data analysis procedure using data from a relative reinforcement efficacy study of simulated marijuana purchasing.     

TOKEN REINFORCEMENT: A REVIEW AND ANALYSIS

Token reinforcement procedures and concepts are reviewed and discussed in relation to general principles of behavior. The paper is divided into four main parts. Part I reviews and discusses previous research on token systems in relation to common behavioral functions—reinforcement, temporal organization, antecedent stimulus functions, and aversive control—emphasizing both the continuities with other contingencies and the distinctive features of token systems. Part II describes the role of token procedures in the symmetrical law of effect, the view that reinforcers (gains) and punishers (losses) can be measured in conceptually analogous terms. Part III considers the utility of token reinforcement procedures in cross‐species analysis of behavior more generally, showing how token procedures can be used to bridge the methodological gulf separating research with humans from that with other animals. Part IV discusses the relevance of token systems to the field of behavioral economics. Token systems have the potential to significantly advance research and theory in behavioral economics, permitting both a more refined analysis of the costs and benefits underlying standard economic models, and a common currency more akin to human monetary systems. Some implications for applied research and for broader theoretical integration across disciplines will also be considered.     

Matching and conditioned reinforcement rate

Attempts to examine the effects of variations in relative conditioned reinforcement rate on choice have been confounded by changes in rates of primary reinforcement or changes in the value of the conditioned reinforcer. To avoid these problems, this experiment used concurrent observing responses to examine sensitivity of choice to relative conditioned reinforcement rate. In the absence of observing responses, unsignaled periods of food delivery on a variable-interval 90-s schedule alternated with extinction on a center key (i.e., a mixed schedule was in effect). Two concurrently available observing responses produced 15-s access to a stimulus differentially associated with the schedule of food delivery (S+). The relative rate of S+ deliveries arranged by independent variable-interval schedules for the two observing responses varied across conditions. The relation between the ratio of observing responses and the ratio of S+ deliveries was well described by the generalized matching law, despite the absence of changes in the rate of food delivery. In addition, the value of the S+ deliveries likely remained constant across conditions because the ratio of S+ to mixed schedule food deliveries remained constant. Assuming that S+ deliveries serve as conditioned reinforcers, these findings are consistent with the functional similarity between primary and conditioned reinforcers suggested by general choice theories based on the concatenated matching law (e.g., contextual choice and hyperbolic value-added models). These findings are inconsistent with delay reduction theory, which has no terms for the effects of rate of conditioned reinforcement in the absence of changes in rate of primary reinforcement.     

The effects of punishment intensity on squirrel monkeys

Responses of squirrel monkeys were maintained by a variable-interval schedule of food reinforcement. Concurrently, punishment consisting of a brief electric shock followed each response. As has been found for pigeons and rats, punishment did not produce extreme, all-or-none reactions. By gradually increasing the punishment intensity it was possible to produce response rates intermediate to no suppression and complete suppression. Similarly, the moment-to-moment response rate was free of extreme fluctuations. A “warm-up” effect occurred in which the punished responses were especially suppressed during the initial part of a session. The pre-punishment performance was negatively accelerated within a session, and punishment reduced the degree of negative acceleration. When punishment was discontinued, responding recovered immediately except when suppression had been complete or prolonged. When the punishment intensity was decreased gradually, more suppression resulted at a given intensity than when intensity was increased gradually. This suggests a “behavioral inertia” effect wherein behavior at a new punishment intensity is biased toward the behavior at the previous value. A corollary generalization is that the larger the change in intensity, the less the behavior at the new value will be biased toward the behavior at the previous value.     

An analysis of contrast effects in multiple schedules

Some phenomena of behavioral contrast in multiple schedules are reviewed, and three accounts of contrast are considered. Rate changes within a constant schedule component (transient contrasts) are distinguished from rate changes across successive schedule cycles (sustained contrasts). Pigeons were exposed to a three-component multiple schedule, in which a stimulus correlated with a constant variable interval schedule of reinforcement was preceded by a stimulus correlated with more frequent variable interval reinforcement, or by an extinction stimulus. If the preceding stimulus was correlated with more frequent reinforcement, the response rate in the constant component was low and increased with time. If the preceding stimulus was correlated with extinction, the rate in the constant component was high and decreased with time. Similar transient contrasts were observed in a two-component multiple schedule with different variable interval schedules in the two components. The transient contrast effects in the three-component schedule were shown to depend on differential reinforcement frequency rather than differential response rate in the preceding component. Such transient contrasts were not sufficient to account for sustained contrast effects observed in these experiments. The relation of these findings to the concepts of excitation and inhibition is discussed.     

A Bayesian formulation of behavioral control

Helplessness, a belief that the world is not subject to behavioral control, has long been central to our understanding of depression, and has influenced cognitive theories, animal models and behavioral treatments. However, despite its importance, there is no fully accepted definition of helplessness or behavioral control in psychology or psychiatry, and the formal treatments in engineering appear to capture only limited aspects of the intuitive concepts. Here, we formalize controllability in terms of characteristics of prior distributions over affectively charged environments. We explore the relevance of this notion of control to reinforcement learning methods of optimising behavior in such environments and consider how apparently maladaptive beliefs can result from normative inference processes. These results are discussed with reference to depression and animal models thereof.     

Toward quantifying the abuse liability of ultraviolet tanning: A behavioral economic approach to tanning addiction

Many adults engage in ultraviolet indoor tanning despite evidence of its association with skin cancer. The constellation of behaviors associated with ultraviolet indoor tanning is analogous to that in other behavioral addictions. Despite a growing literature on ultraviolet indoor tanning as an addiction, there remains no consensus on how to identify ultraviolet indoor tanning addictive tendencies. The purpose of the present study was to translate a behavioral economic task more commonly used in substance abuse to quantify the "abuse liability" of ultraviolet indoor tanning, establish construct validity, and determine convergent validity with the most commonly used diagnostic tools for ultraviolet indoor tanning addiction (i.e., mCAGE and mDSM‐IV‐TR). We conducted a between‐groups study using a novel hypothetical Tanning Purchase Task to quantify intensity and elasticity of ultraviolet indoor tanning demand and permit statistical comparisons with the mCAGE and mDSM‐IV‐TR. Results suggest that behavioral economic demand is related to ultraviolet indoor tanning addiction status and adequately discriminates between potential addicted individuals from nonaddicted individuals. Moreover, we provide evidence that the Tanning Purchase Task renders behavioral economic indicators that are relevant to public health research. The present findings are limited to two ultraviolet indoor tanning addiction tools and a relatively small sample of high‐risk ultraviolet indoor tanning users; however, these pilot data demonstrate the potential for behavioral economic assessment tools as diagnostic and research aids in ultraviolet indoor tanning addiction studies.     

Quantification of rats' behavior during reinforcement periods

What is treated as a single unit of reinforcement often involves what could be called a reinforcement period during which two or more acts of ingestion may occur, and each of these may have associated with it a series of responses, some reflexive, some learned, that lead up to ingestion. Food-tray presentation to a pigeon is an example of such a “reinforcement period.” In order to quantify this behavior, a continuous-reinforcement schedule was used as the reinforcement period and was chained to a fixed-ratio schedule. Both fixed-ratio size and reinforcement-period duration were manipulated. Rats were used as subjects, food as reinforcement, and a lever press as the operant. Major findings included (a) a rapid decline in response rates during the first 15 to 20 seconds of the reinforcement periods, and (b) a strong positive relationship between these response rates and the size of the fixed ratio. Also revealed was a short scallop not normally found in fixed-ratio response patterns, whose length was a function of fixed-ratio size and reinforcement-period duration. It is suggested that rapidly fluctuating excitatory processes can account for many of these findings and that such processes are functionally significant in terms of behavioral compensation.     

The role of contingencies and "principles of behavioral variation" in pigeons' pecking

Staddon and Simmelhag's proposal that behavior is produced by “principles of behavioral variation” instead of contingencies of reinforcement was tested in two experiments. In the first experiment pigeons were exposed to either a fixed-interval schedule of response-contingent reinforcement, an autoshaping schedule of stimulus-contingent reinforcement, or a fixed-time schedule of noncontingent reinforcement. Pigeons exposed to contingent reinforcement came to peck more rapidly than those exposed to noncontingent reinforcement. Staddon and Simmelhag's “principles of behavioral variation” included the proposal that patterns (interim and terminal) were a function of momentary probability of reinforcement. In the second experiment pigeons were exposed to either a fixed-time or a random-time schedule of noncontingent reinforcement. Pecking showed a constant frequency of occurrence over postfood time on the random-time schedule. Most behavior showed patterns on the fixed-time schedule that differed in overall shape (i.e., interim versus terminal) from those shown on the random-time schedule. It was concluded that both the momentary probability of reinforcement and postfood time can affect patterning.