Choice with probabilistic reinforcement: effects of delay and conditioned reinforcers.

Two experiments measured pigeons' choices between probabilistic reinforcers and certain but delayed reinforcers. In Experiment 1, a peck on a red key led to a 5-s delay and then a possible reinforcer (with a probability of .2). A peck on a green key led to a certain reinforcer after an adjusting delay. This delay was adjusted over trials so as to estimate an indifference point, or a duration at which the two alternatives were chosen about equally often. In all conditions, red houselights were present during the 5-s delay on reinforced trials with the probabilistic alternative, but the houselight colors on nonreinforced trials differed across conditions. Subjects showed a stronger preference for the probabilistic alternative when the houselights were a different color (white or blue) during the delay on nonreinforced trials than when they were red on both reinforced and nonreinforced trials. These results supported the hypothesis that the value or effectiveness of a probabilistic reinforcer is inversely related to the cumulative time per reinforcer spent in the presence of stimuli associated with the probabilistic alternative. Experiment 2 tested some quantitative versions of this hypothesis by varying the delay for the probabilistic alternative (either 0 s or 2 s) and the probability of reinforcement (from .1 to 1.0). The results were best described by an equation that took into account both the cumulative durations of stimuli associated with the probabilistic reinforcer and the variability in these durations from one reinforcer to the next.     

Behavioral economics

Economics, like behavioral psychology, is a science of behavior, albeit highly organized human behavior. The value of economic concepts for behavioral psychology rests on (1) their empirical validity when tested in the laboratory with individual subjects and (2) their uniqueness when compared to established behavioral concepts. Several fundamental concepts are introduced and illustrated by reference to experimental data: open and closed economies, elastic and inelastic demand, and substitution versus complementarity. Changes in absolute response rate are analyzed in relation to elasticity and intensity of demand. The economic concepts of substitution and complementarity are related to traditional behavioral studies of choice and to the matching relation. The economic approach has many implications for the future of behavioral research and theory. In general, economic concepts are grounded on a dynamic view of reinforcement. The closed-economy methodology extends the generality of behavioral principles to situations in which response rate and obtained rate of reinforcement are interdependent. Analysis of results in terms of elasticity and intensity of demand promises to provide a more direct method for characterizing the effects of “motivational” variables. Future studies of choice should arrange heterogeneous reinforcers with varying elasticities, use closed economies, and modulate scarcity or income. The economic analysis can be extended to the study of performances that involve subtle discriminations or skilled movements that vary in accuracy or quality as opposed to rate or quantity, and thus permit examination of time/accuracy trade-offs.     

Graded differential reinforcement: Response-dependent reinforcer amount

After key pecking had been autoshaped, six pigeons were exposed to a condition in which the duration of grain availability at the end of an 8-second trial depended on the number of responses emitted during the trial (0.25-second access to grain per response). This procedure, called correlated reinforcement, alternated across conditions with the automaintenance baseline in which the 8-second trial terminated with a constant 2.5-second access to grain. Two control procedures were run; in both, the reinforcer durations were yoked to those obtained in the last correlated session. In the yoked control no responses were required, but in the single-response yoked control at least one response was required to receive the yoked duration. The correlated condition maintained response rates above those produced by the two control conditions. These results may be accounted for by differential reinforcement.     

Confirmation of linear system theory prediction: Changes in Herrnstein's k as a function of changes in reinforcer magnitude

Eight human subjects pressed a lever on a range of variable-interval schedules for 0.25¢ to 35.0¢ per reinforcement. Herrnstein's hyperbola described seven of the eight subjects' response-rate data well. For all subjects, the y-asymptote of the hyperbola increased with increasing reinforcer magnitude and its reciprocal was a linear function of the reciprocal of reinforcer magnitude. These results confirm predictions made by linear system theory; they contradict formal properties of Herrnstein's account and of six other mathematical accounts of single-alternative responding.     

An examination of stimulus technology level and preference displacement during multiple stimulus without replacement preference assessments

Researchers have observed that preference for edible items may displace preference for leisure items when items from the stimulus classes are assessed together within multiple stimulus without replacement (MSWO) preference assessments. The current study extends previous research by examining patterns of preference and displacement when assessing preference for high-tech and low-tech leisure items. We conducted three separate MSWO preference assessments; one assessing low-tech items, another assessing high-tech items, and a combined assessment (using the top four high- and low-tech items from the previous assessments) to test for displacement. Preference for high-tech items fully or partially displaced preference for low-tech items in five of eight participants. We then conducted concurrent and single operant reinforcer assessments using the highly preferred high-tech and low-tech items. Reinforcer assessment results demonstrated similar levels of responding for the highest preferred high-tech and low-tech items, indicating that combining high- and low-tech items within preference assessments may influence the validity of results.     

The use of mixed modeling to evaluate the impact of treatment integrity on learning

Treatment integrity, or the degree to which an intervention is implemented as intended, is a critical feature of skill acquisition tasks. Single‐case design consistently demonstrates that low treatment integrity slows or inhibits learning, but the relative impact of different types of instructional errors, or the presence of multiple errors, is less clear. The present study utilized a multilevel modeling approach to evaluate the impact of type of error (omission versus omission and commission) and error component (reinforcer delivery versus feedback) on learning. Findings revealed that learning outcomes worsened based on the type of error and as the complexity of errors increased; more specifically, participants performed better when only a single type of error occurred and when only a single error component was manipulated. Additionally, individual characteristics contributed to learning outcomes, highlighting the use of multilevel modeling as a helpful tool to supplement single‐case design. The differential impact of integrity errors on learning may be due to timing of errors (i.e., commission errors more likely to occur early in learning) or how errors affect reinforcement schedule versus discriminative control.     

All Behavior is choice: Revisiting an evolutionary theory's account of behavior on single schedules

The axiomatic principle that all behavior is choice was incorporated into a revised implementation of an evolutionary theory's account of behavior on single schedules. According to this implementation, target responding occurs in the context of background responding and reinforcement. In Phase 1 of the research, the target responding of artificial organisms (AOs) animated by the revised theory was found to be well described by an exponentiated hyperbola, the parameters of which varied as a function of the background reinforcement rate. In Phase 2, the effect of reinforcer magnitude on the target behavior of the AOs was studied. As in Phase 1, the AOs' behavior was well described by an exponentiated hyperbola, the parameters of which varied with both the target reinforcer magnitude and the background reinforcement rate. Evidence from experiments with live organisms was found to be consistent with the Phase-1 predictions of the revised theory. The Phase-2 predictions have not been tested. The revised implementation of the theory can be used to study the effects of superimposing punishment on single-schedule responding, and it may lead to the discovery of a function that relates response rate to both the rate and magnitude of reinforcement on single schedules.     

Replacing relative reinforcing efficacy with behavioral economic demand curves

Relative reinforcing efficacy refers to the behavior-strengthening or maintaining property of a reinforcer when compared to that of another reinforcer. Traditional measures of relative reinforcing efficacy sometimes have led to discordant results across and within studies. By contrast, previous investigations have found traditional measures to be congruent with behavioral economic measures, which provide a framework for integrating the discordant results. This study tested whether the previously demonstrated congruence between traditional relative reinforcing efficacy measures and behavioral economic demand curve measures is sufficiently robust to persist when demand for one reinforcer is altered. Cigarette smokers pulled plungers for cigarettes or two magnitudes of money on progressive-ratio schedules that increased the response requirement across sessions. Demand for the two different reinforcers was assessed in single-schedule and concurrent-schedule sessions. Demand curve measures Pmax and Omax correlated significantly with traditional measures of breakpoint and peak response rate, respectively. Relative locations of demand curves for money and cigarettes under single schedules predicted preference in concurrent schedules in most cases. Although measures of relative reinforcing efficacy for money changed with money magnitude, the congruence between traditional and behavioral economic measures remained intact. This robust congruence supports the proposal that demand curves should replace measures of relative reinforcing efficacy. The demand curve analysis illustrates why concordance between traditional measures is expected under some experimental conditions but not others.     

A brief computer-based assessment of reinforcer dimensions affecting choice

In an extension of Neef, Shade, and Miller (1994), we used a brief computer-based assessment of differential responsiveness to reinforcer rate, quality, delay, and response effort in affecting the choices of 11 participants. The assessment involved successive presentations of two concurrent sets of math problems, each set associated with competing reinforcer or response dimensions in a counterbalanced fashion. The results showed that the reinforcer and response dimensions differentially affected choice, with time-allocation patterns varying across students.