On the measurement of reinforcement frequency in the study of preference

In a two-link, concurrent-chain schedule, pigeons' pecks on each key during the initial link occasionally produced a terminal link, during which only that key was operative. Responses in the terminal link were reinforced with food on either fixed-interval or variable-interval schedules. In one experiment, relative amount of responding in the initial link equaled the relative harmonic rate of reinforcement in the terminal links. In a second experiment, the selection of interreinforcement intervals in variable-interval schedules in the terminal links was such that rates of reinforcement based on the harmonic or on the arithmetic means of the interreinforcement intervals predicted opposite preferences in the initial links. The observed preference was consistent with that predicted by the harmonic rather than by the arithmetic rates of reinforcement.     

Responding on concurrent-chains schedules in open and closed economies.

Pigeons' key pecks were reinforced according to concurrent-chains schedules of reinforcement. The programmed average time from the onset of the initial links to a terminal link entry was held constant across conditions while the value of variable-interval schedules in the terminal links was varied. Performance was assessed under two economic conditions: (a) an open economy in which session duration was limited to 1 hr and subjects were maintained at 80% of their free-feeding body weights with postsession food when necessary; and (b) a closed economy in which sessions were 23.5 hr long and no deprivation regimen was in effect. In all cases, the relative rate of responding in the initial links matched the reduction in overall delay to primary reinforcement correlated with entry into one terminal link relative to the reduction in delay correlated with entry into the other terminal link. Although the sum of responses made in the initial links and terminal links was found to increase, then decrease, as the rate of food presentation decreased in the closed economy, there was no consistent effect of overall rate of food presentation on total responding in the open economy. The choice data suggest that relative delay reduction predicts choice accurately, regardless of economic context.     

Generalization gradients following two-response discrimination training

Stimulus generalization was investigated using institutionalized human retardates as subjects. A baseline was established in which two values along the stimulus dimension of auditory frequency differentially controlled responding on two bars. The insertion of the test probes disrupted the control established to the two S^Ds during training. The discrimination was recovered between each test probe and the resulting gradients were stable across 10 test sessions. These gradients, supported by other two-response generalization studies, indicate that this type of two-response discrimination training divides the stimulus dimension into two functional classes separated by a region of transition from one class to the other. Each stimulus value in a class, which extends from an S^D outward to the functional limit of the dimension, controls a similar proportion of the two responses as each other value in the class. All values on the stimulus dimension control identical response rates with an absence of the usual generalization decrement. The latency of the initial response, however, shows a bimodal gradient with the modes at the S^D values.     

Nicotine vs. ethanol discrimination: Extinction and spontaneous recovery of responding

Studies regarding extinction and spontaneous recovery of the discriminative stimulus effects of drugs are limited. Eight rats were initially trained to discriminate nicotine (0.4 mg/kg) vs. ethanol (800 mg/kg). For four rats, itraperitaneal (IP) administrations of nicotine fifteen minutes prior to fifteen-minute training sessions served as a discriminative stimulus (S^D) for predicting food-reinforced lever pressing (VI-1 min). On other sessions ethanol functioned in predicting nonreinforcement (S^Δ). The stimulus roles of the drugs were counterbalanced for the remaining four rats. S^Δ and S^D sessions alternated quasi-randomly with two daily sessions at 1000 and 1400 hours. Discriminative control was not disrupted following ten extinction sessions under a non-drug/saline condition, but was disrupted following extinction sessions under the original training drugs. Instances of spontaneous recovery (SR) occurred throughout extinction under the drug condtions. There was no evidence for SR two weeks following extinction, but partial recovery four weeks following the final extinction phase. Contextual status (context renewal) had neither a restorative or disruptive impact on extiguished or discriminated responding, respectively. Theser results support and extend the limited number of other studies by demonstrating extinction and spontaneous recovery of responding discriminated bytwo distinct drugs. Some theoretical interpretations regarding history effects and training in the context of drug discrimination are entertained.     

Resistance to change produced by access to fixed-delay versus variable-delay terminal links

Pigeons' responding was reinforced on a multiple schedule consisting of two two-link chain schedules presented in regular alternation. Responding in initial links (always variable-interval 60-s) produced a key-color change and access to a terminal link. The terminal link for one chain provided food after a fixed delay (fixed-interval or fixed-time); the terminal link for the other provided food after a variable delay (variable-interval or variable-time). The average duration of the terminal-link schedules was varied across conditions, but in every condition the arithmetic mean of the variable-delay terminal-link schedule was equal to the duration of the fixed delay. Response rates were higher in the initial links of the chains with the variable-delay terminal links. Response-decreasing operations (satiation, extinction) were used after performances reached asymptote. Response rates maintained by access to variable-delay terminal links tended to be more resistant to change than were rates maintained by access to fixed-delay terminal links. These results are consistent with the preference for variable- over fixed-interval terminal links observed with concurrent-chains schedules, suggesting (1) that immediacy of reinforcement influences the conditioned reinforcing potency of access to a terminal link and (2) that choice in concurrent chains and resistance of responding to change may be manifestations of the same effect of reinforcement.     

Development of behavioral compensation to the effects of scopolamine during fixed-interval reinforcement

Rats were injected with scopolamine before every daily session of water reinforcement on a fixed-interval (FI) schedule. Initially the drug decreased the rate of responding. Control injections of scopolamine following each session did not. Over 119 sessions, the typical FI performance developed more slowly in the animals drugged before the sessions. Their rates of responding increased from session to session, to a level slightly greater than that of the animals drugged after the sessions. Their rates did not increase. The effects of injections before the session were not duplicated by increasing the deprivation of animals drugged after the session.     

Discriminative functions based on a delay in the reinforcement relation

Pigeons were shown to come under discrimination control when the S^D and S^Δ were temporally separated from reinforcement and non-reinforcement. S^D and S^Δ consisted of distinctive key illuminations presented separately. Responding on an FR 5 in the presence of S^D or S^Δ produced a third stimulus containing a schedule requirement. If this third (or interpolated) stimulus was preceded by S^D, responding in its presence produced reinforcement followed by a time-out (TO). If, on the other hand, the third stimulus was preceded by S^Δ, responding produced TO alone. In this fashion, the same stimulus and the same response requirement were imposed between S^D and the reinforcement as between S^Δ and the TO. In Experiment I, the schedule employed during the interpolated stimulus was FR; in Experiment II, FI. Discrimination reversal was accomplished in both experiments.     

Pigeons' preferences for stimulus information: effects of amount of information

A concurrent-chain procedure was used to study pigeons' preferences as a function of amount of information. Pigeons chose between two terminal links. Both terminal links ended in food reinforcement with probability (p) and in blackout with probability (1–p). One terminal link (noninformative link) was signalled by a stimulus uncorrelated with either food or blackout. The other terminal link (informative link) was signalled by stimuli correlated with these outcomes. Amount of information conveyed by these stimuli was varied across conditions by changing the probability of reinforcement (p) and blackout (1–p). The pigeons strongly preferred the informative link, and preferences were greater at p values above 0.50 than for their complements. The pigeons engaged in different behaviors during the stimulus periods, suggesting that the value of informative stimuli may be in their function as discriminative stimuli for interim activities and terminal responses.     

The effect of multiple S periods on responding on a fixed-interval schedule: IV. Effect of continuous S with only short S probes

Pigeons were studied under FI 500 sec in which an S^Δ was present throughout the interval except during the terminal 50-sec segment and one earlier 50-sec segment. Very little responding occurred during the presence of S^Δ. The rate of responding in the earlier 50-sec S^D segments was lower than in the terminal S^D segment. There was a clear trend for the rate of responding in the earlier S^D segment to be progressively higher the later it occurred in the course of the FI 500 sec. This trend was shown roughly to parallel the increasing rate of responding in a conventional FI 500 sec with no interruption by S^Δ. Since the changing tendency to respond through the FI survives massive disruption by S^Δ, it is concluded that the control of responding through the FI does not require continuous mediating behavior. It is suggested that it is the decaying retroactive influence of the reinforcer on responses that occurred longer and longer before the reinforcer occurred which produces the familiar scalloped pattern of responding under FI schedules.     

Solution of the intermediate size problem by pigeons

Pigeons learned to respond to the middle-sized member (S^D) of a set of three simultaneously presented stimuli with responses to the S^D reinforced on a VI 1 schedule. They were then tested for several days with other sets of three stimuli. One procedure presented reinforcements on a VI 1 schedule during the test independent of the stimulus chosen when a reinforcement was programmed. The tests were also given under extinction conditions. With the testing carried out with extinction, preference consistently was for the test stimulus most similar in physical size to the S^D. However, when the tests were with reinforcement, random responding resulted. Another effect of testing with reinforcement was an increase in incorrect responding with the training set. Such a test procedure was unsatisfactory for determining the effective aspect of the S^D. The conclusion, based on the data of the extinction series, was that pigeons learned the intermediate size problem on the basis of the discrimination of absolute stimulus properties.     

On the effects of noncontingent food delivery during naturally occurring periods of deprivation and satiation

Four adults diagnosed with moderate to profound mental retardation performed a manual response that was reinforced with food identified from a stimulus preference assessment. During baseline, the response was reinforced on a variable ratio (VR) schedule. Participants were then exposed to noncontingent reinforcement (NCR) plus extinction, and no‐food (i.e., extinction) conditions. A combination multielement and reversal design was used to evaluate intervention effects. For each participant, sessions were conducted both before and after the midday meal during baseline and NCR‐plus‐extinction conditions, thus capitalizing on naturally occurring states of food deprivation and satiation. Results showed that response rates were slightly higher during deprivation sessions than during satiation sessions during NCR‐plus‐extinction and VR schedules for three of the four participants. For three participants, initial NCR schedules did not reduce responding; however, subsequent NCR schedules, which were twice as dense, were effective in reducing response rates. The results are discussed in terms of the development of NCR as a reductive technology and the manipulation of establishing operations applied to the habilitation of individuals with developmental disabilities. The use of a basic experimental preparation as a method of examining decelerative interventions is also addressed. Copyright © 2000 John Wiley & Sons, Ltd.     

Effects of experience on preference between forced and free choice

Preference between forced choice and free choice in concurrent-chain schedules of reinforcement was investigated in pigeons after exposure to particular combinations of terminal links. In Experiment 1, in which terminal links always ended with reinforcers, one of three pairs of terminal links was arranged as preexposure: (a) both terminal links had only one key (forced choice), (b) both terminal links had two keys (free choice), or (c) a combination of forced and free choice was arranged across sessions. In test sessions following the preexposure, pigeons' preferences rapidly shifted to the terminal links with which they had no recent experience. In Experiment 2, the same procedure was repeated except that each terminal link ended intermittently with reinforcers with a probability of .5 and there was no terminal-link arrangement with a combination of free and forced choice. Pigeons showed the same preference changes as in Experiment 1, but the preference changes did not appear immediately at the beginning of test sessions. These data suggest that recent previous experience was a more important determinant of preference than the difference between forced-choice and free-choice terminal links.     

Discrimination learning as a function of stimulus location along an auditory intensity continuum

Eight groups of rats were trained on an auditory intensity discrimination in which the discriminative stimuli were separated by 10 decibels (db). Four pairs of stimuli were selected from different regions along a 60–100 db (SPL) intensity continuum. Counterpart groups were trained on each stimulus pair, with the relative intensity positions of the reinforced stimulus (S^D) and the non-reinforced stimulus (S^Δ) reversed for the two groups. Discrimination acquisition curves were compared to determine whether stimuli separated by equal logarithmic units were of comparable “difficulty”, and to determine the relative effectiveness of an S^D serving as the more versus less intense member of a stimulus pair. It was concluded that: (1) When S^D is the more intense, auditory intensities of constant logarithmic separation are graded in “difficulty” along the intensity continuum; high intensity discriminative stimuli are most readily discriminated. When S^Δ is the more intense, this graded effect is not evident. (2) For a given continuum location, discrimination is inferior when S^Δ is the more intense. This effect is most pronounced at the high intensity end of the continuum and is chiefly attributable to differences in the rate of S^Δ responding.     

Punishment by SD associated with fixed-ratio reinforcement

Two pigeons were trained with positive reinforcement on a multiple FR VI 2 schedule. The VI 2 component was held constant while the FR component was changed from ratios of 1 to 300. After responding had stabilized at each FR value, VI responses produced briefly either the fixed-ratio S^D or a stimulus uncorrelated with either schedule component. Compared to the effects of the uncorrelated stimulus change, the fixed-ratio S^D produced a decrease in VI responding proportional to the size of the FR requirement. It is concluded that stimuli associated with high FR schedules served as punishment for the ongoing behavior.     

Effects of food deprivation and reinforcement magnitude on conditioned suppression

In Experiment I, the responding of rats lever pressing on a variable-interval schedule for sucrose solution was partially suppressed by a variable duration conditioned stimulus followed by shock. When food deprivation was increased, response rates during and before the conditioned stimulus increased monotonically. Varying the concentration of sucrose across blocks of sessions or from session to session in a semi-random sequence had little effect on response rates either before or during the conditioned stimulus. With a fixed sequence of increasing concentrations across a five-session block, increased concentration produced much more rapid increases in response rates before than during the conditioned stimulus. In Experiment II, rats were presented with the same sequence of increasing concentrations across a five-session block. When tested at 80% body weight, response rates increased rapidly as concentration increased, but at 100%, body-weight rates increased only slightly. The effect of a change in body weight in Experiment II thus mimicked the effect of the conditioned stimulus in the latter part of Experiment I. These findings support the view that the effect of a pre-aversive conditioned stimulus is similar to that of a change in food deprivation, but unlike that of a change in reinforcement magnitude.     

Behavioral effects of pairing an SD with a decreasing limited-hold reinforcement schedule

Four pigeons were trained on a multiple reinforcement schedule consisting of two limited-hold schedules, one in which a discriminative stimulus (S^D) accompanied the periodic reinforcement contingency, and one in which the discriminative stimulus was omitted. The duration of the limited-hold in each component of the multiple schedule was reduced in parallel steps. It was shown that behavioral differences between the two schedules were attenuated by this manipulation of temporal parameters. When S^D was reduced in duration, three out of four pigeons responded with extremely high S^Δ rates, despite the regular pairing of S^Δ with the reinforcement contingency. These high rates qualitatively resembled the rapid rates emitted on the analogous no-S^D component.     

Drug discrimination using a conditioned taste-aversion paradigm in rhesus monkeys.

The development of drug discrimination was assessed in rhesus monkeys using the conditioned taste-aversion paradigm. Monkeys were initially trained to respond under a fixed-ratio 30-response schedule of food-pellet delivery to assess the rate-decreasing effects of alprazolam (0.03 to 3 mg/kg, i.m., 60 min presession). Alprazolam decreased responding at doses greater than 0.1 mg/kg. Discriminative stimulus effects of alprazolam were then assessed by giving 0.03 mg/kg before sessions in which 1.8 mEq/kg lithium chloride was given immediately after the session (alprazolam/lithium session). On intervening days, saline was given before and after the session (saline/saline session). Rates of responding decreased over successive alprazolam/lithium sessions and also during the saline/saline session that immediately followed an alprazolam/lithium session. During subsequent saline/saline sessions, rates of responding returned to levels near baseline rates within two to four sessions. The discriminative stimulus effects of alprazolam were then assessed by giving 0.1 mg/kg before sessions in which 1 mg/kg d-amphetamine was given immediately after the session (alprazolam/d-amphetamine session). Rates of responding decreased during subsequent alprazolam/d-amphetamine sessions in drug-experienced monkeys, but did not decrease during intervening saline/saline sessions. These findings demonstrate that drug stimuli associated with postsession drug injections can rapidly develop control over behavior and suggest that similar methods be explored in the assessment of drug discrimination.     

Comparing preference and resistance to change in constant- and variable-duration schedule components

Two experiments explored preference and resistance to change in concurrent chains in which the terminal links were variable-interval schedules that ended either after a single reinforcer had been delivered (variable duration) or after a fixed period of access to the schedule (constant duration). In Experiment 1, pigeons' preference between the same pair of terminal links overmatched relative reinforcement rate when the terminal links were of constant duration, but not when they were of variable duration. Responding during the richer terminal link decreased less, relative to baseline, when response-independent food was presented during the initial links according to a variable-time schedule. In Experiment 2, all subjects consistently preferred a terminal link that consisted of 20-s access to a variable-interval 20-s schedule over a terminal link that ended after one reinforcer had been delivered by the same schedule. Results of resistance-to-change tests corresponded to preference, as responding during the constant-duration terminal link decreased less, relative to baseline, when disrupted by both response-independent food during the initial links and prefeeding. Overall, these data extend the general covariation of preference and resistance to change seen in previous studies. However, they suggest that reinforcement numerosity, including variability in the number of reinforcers per terminal-link entry, may sometimes affect preference and resistance to change in ways that are difficult to explain in terms of current models.     

Antecedent reinforcement contingencies in the stimulus control of an auditory discrimination

In order to assess possible confounding of discriminative stimulus effects with those produced by the reinforcing stimulus, three groups of four rats each were trained for 45 hr on a variable-interval 1-min reinforcement program. Two groups were run on a multiple variable-interval extinction schedule in which the reinforcement stimulus (S^D) and the nonreinforcement stimulus (S^Δ) were two intensities of a 4-kHz (cps) tone separated by 40 or 10 db. The third group was run on a mixed schedule with a single intensity constantly present. The mixed-schedule animals showed no discrimination of the reinforcement program. Under the multiple schedule, the highest S^Δ rates were obtained after S^D intervals, regardless of the reinforcement availability in the S^D interval. These local rate variations in S^Δ were small in proportion to those produced by the S^D versus S^Δ intensities.     

Economics, ecologics, and mechanics: The dynamics of responding under conditions of varying motivation

The mechanics of behavior developed by Killeen (1994) is extended to deal with deprivation and satiation and with recovery of arousal at the beginning of sessions. The extended theory is validated against satiation curves and within-session changes in response rates. Anomalies, such as (a) the positive correlation between magnitude of an incentive and response rates in some contexts and a negative correlation in other contexts and (b) the greater prominence of incentive effects when magnitude is varied within the session rather than between sessions, are explained in terms of the basic interplay of drive and incentive motivation. The models are applied to data from closed economies in which changes of satiation levels play a key role in determining the changes in behavior. Relaxation of various assumptions leads to closed-form models for response rates and demand functions in these contexts, ones that show reasonable accord with the data and reinforce arguments for unit price as a controlling variable. The central role of deprivation level in this treatment distinguishes it from economic models. It is argued that traditional experiments should be redesigned to reveal basic principles, that ecologic experiments should be redesigned to test the applicability of those principles in more natural contexts, and that behavioral economics should consist of the applications of these principles to economic contexts, not the adoption of economic models as alternatives to behavioral analysis.