Effects of experience on preference between forced and free choice

Preference between forced choice and free choice in concurrent-chain schedules of reinforcement was investigated in pigeons after exposure to particular combinations of terminal links. In Experiment 1, in which terminal links always ended with reinforcers, one of three pairs of terminal links was arranged as preexposure: (a) both terminal links had only one key (forced choice), (b) both terminal links had two keys (free choice), or (c) a combination of forced and free choice was arranged across sessions. In test sessions following the preexposure, pigeons' preferences rapidly shifted to the terminal links with which they had no recent experience. In Experiment 2, the same procedure was repeated except that each terminal link ended intermittently with reinforcers with a probability of .5 and there was no terminal-link arrangement with a combination of free and forced choice. Pigeons showed the same preference changes as in Experiment 1, but the preference changes did not appear immediately at the beginning of test sessions. These data suggest that recent previous experience was a more important determinant of preference than the difference between forced-choice and free-choice terminal links.     

The effect of conditioned reinforcement rate on choice: a review

We review the nature of conditioned reinforcement, including evidence that conditioned reinforcers maintain choice behavior in concurrent schedules and that they elevate responding in the terminal links of concurrent‐chains schedules. A question has resurfaced recently: Do theories of choice in concurrent‐chains schedules need to include a term reflecting greater preference for higher rates of conditioned reinforcement? The review of several studies addressing this point suggests that such a term is inappropriate. Elevated rates of conditioned reinforcement (and responding) in the terminal links of concurrent‐chains schedules do not lead to greater preference in the initial link leading to the higher rate of conditioned reinforcement. If anything, the opposite preference is likely to occur. This result is not surprising, since the additional putative conditioned reinforcers in the terminal link are not correlated with a reduction in time to primary reinforcement nor with an increase in value.     

Molar versus local reinforcement probability as determinants of stimulus value.

During one component of a multiple schedule, pigeons were trained on a discrete-trial concurrent variable-interval variable-interval schedule in which one alternative had a high scheduled rate of reinforcement and the other a low scheduled rate of reinforcement. When the choice proportion between the alternatives matched their respective relative reinforcement frequencies, the obtained probabilities of reinforcement (reinforcer per peck) were approximately equal. In alternate components of the multiple schedule, a single response alternative was presented with an intermediate scheduled rate of reinforcement. During probe trials, each alternative of the concurrent schedule was paired with the constant alternative. The stimulus correlated with the high reinforcement rate was preferred over that with the intermediate rate, whereas the stimulus correlated with the intermediate rate of reinforcement was preferred over that correlated with the low rate of reinforcement. Preference on probe tests was thus determined by the scheduled rate of reinforcement. Other subjects were presented all three alternatives individually, but with a distribution of trial frequency and reinforcement probability similar to that produced by the choice patterns of the original subjects. Here, preferences on probe tests were determined by the obtained probabilities of reinforcement. Comparison of the two sets of results indicates that the availability of a choice alternative, even when not responded to, affects the preference for that alternative. The results imply that models of choice that invoke only obtained probability of reinforcement as the controlling variable (e.g., melioration) are inadequate.     

Caregiver preference for reinforcement‐based interventions for problem behavior maintained by positive reinforcement

Social validity of behavioral interventions typically is assessed with indirect methods or by determining preferences of the individuals who receive treatment, and direct observation of caregiver preference rarely is described. In this study, preferences of 5 caregivers were determined via a concurrent‐chains procedure. Caregivers were neurotypical, and children had been diagnosed with developmental disabilities and engaged in problem behavior maintained by positive reinforcement. Caregivers were taught to implement noncontingent reinforcement (NCR), differential reinforcement of alternative behavior (DRA), and differential reinforcement of other behavior (DRO), and the caregivers selected interventions to implement during sessions with the child after they had demonstrated proficiency in implementing the interventions. Three caregivers preferred DRA, 1 caregiver preferred differential reinforcement procedures, and 1 caregiver did not exhibit a preference. Direct observation of implementation in concurrent‐chains procedures may allow the identification of interventions that are implemented with sufficient integrity and preferred by caregivers.     

Freedom and knowledge: an experimental analysis of preference in pigeons

Relative responding in initial links of concurrent-chain schedules showed that pigeons preferred free to forced choices and informative to uninformative stimuli. Variable-interval initial links on two lower keys (white) of a six-key chamber produced terminal links on either two upper-left keys (blue and/or amber) or two upper-right keys (green and/or red). Terminal.links in which pecks on either of two lit keys produced fixed-interval reinforcement (free choice) were preferred to links with only one lit fixed-interval key available (forced choice). Terminal links with different key colors correlated with concurrent fixed-interval reinforcement and extinction (informative stimuli) were preferred to links with these schedules operating on same-color keys (uninformative stimuli). Scheduling extinction for one of the two free-choice keys assessed preference for two lit keys over one lit key, but confounded number with whether stimuli were informative. Fixed-interval reinforcement for both keys in each terminal link, but with different-color keys in one link and same-color keys in the other, showed that preference for informative stimuli did not depend on stimulus variety. Preferences were independent of relative responses per reinforcement and other properties of terminal-link performance.     

Reinforcement and punishment effects in concurrent schedules: A test of two models

The joint effects of punishment and reinforcement on the pigeon's key-peck response were examined in three choice experiments conducted to compare predictions of Farley and Fantino's (1978) subtractive model with those made by Deluty's (1976) and Deluty and Church's (1978) model of punishment. In Experiment 1, the addition of equal punishment schedules to both alternatives of a concurrent reinforcement schedule enhanced the preference exhibited for the more frequent reinforcement alternative. Experiment 2 demonstrated decreases in the absolute response rate for each member of a concurrent reinforcement schedule when increasing frequencies of punishment were added to each alternative. Experiment 3 found that preference for the denser of two reinforcement schedules diminished when the absolute frequencies of reinforcement were increased by a constant factor and conditions of punishment for both alternatives were held constant. Diminished preferences were obtained regardless of whether the frequency of punishment associated with the denser reinforcement schedule was greater or less than that associated with the lean reinforcement alternative. The results from all three experiments uniquely supported Farley and Fantino's (1978) subtractive model of punishment and reinforcement.     

Pigeons' preference for free choice: number of keys versus key area.

In concurrent-chains schedules, pigeons prefer terminal links that provide two keys correlated with reinforcers (free choice) over those that provide only one key (forced choice), terminal-link reinforcement rates being equal. With same-size keys, free choice provides a larger area available for pecking. Preferences were examined using terminal links that differed in key number only (one or two) or key size only (small and medium or medium and large), or that equated the area of the two free-choice keys with that of the forced-choice key. Medium (standard) keys were typically preferred to small keys, but indifference was typically obtained between medium and large keys. The size preference usually overrode free-choice preference with one medium key pitted against two small keys, but free-choice preference was reliably observed with one large key pitted against two medium keys. In other words, preferences were a joint function of key number and key area, implying that free-choice preference is not reducible to preference for larger key areas. Free-choice preference requires separate keys rather than larger areas; the relevant behavioral units are the discriminated operants correlated with each terminal-link key rather than classes defined by topographical features such as area or perimeter.     

Rapid determinations of preference in multiple concurrent-chain schedules.

With concurrent chains arranged for a pigeon's key pecks, pecks on two concurrently available initial-link keys (left and right) respectively produce separately operating terminal links (A and B). Preferences for terminal link A over terminal link B are usually calculated as deviations of relative initial-link response rates (left divided by total pecks) from those during baseline conditions, when A equals B. Baseline preferences, however, are often variable and typically are determined indirectly (e.g., with unequal A and B, reversing left-right assignments of A and B over sessions and estimating the baseline from differences between the relative rates generated). Multiple concurrent-chain schedules, with components each consisting of a pair of concurrent chains, speed the determination of preferences by arranging A and B and their reversal within sessions. In two experiments illustrating the feasibility of this procedure, one component operated with circles projected on initial-link keys and the other with pluses; when left and right initial-link pecks respectively produced terminal links A and B in one component, they produced B and A in the other. Even as the baselines fluctuated, preference was observable within sessions as the difference between relative initial-link response rates in the two components. The first experiment demonstrated the rapid development of preferences when terminal links A and B consisted of fixed-interval 15-s and 30-s schedules. The second demonstrated the sensitivity of the procedure to preference for a fixed-interval 30-s schedule operating for pecks on either of two keys (free choice) over its operating for pecks on only a single key (forced choice).     

Context effects on choice.

Four pigeons responded on a concurrent-chains schedule in four experiments that examined whether the effectiveness of a stimulus as a conditioned reinforcer is best described by a global approach, as measured by the average interreinforcement interval, or by a local contextual approach, as measured by the onset of the stimulus preceding the conditioned reinforcer. The interreinforcement interval was manipulated by the inclusion of an intertrial interval, which increased the overall time to reinforcement but did not change the local contingencies on a given trial A global analysis predicted choice for the richer alternative to decrease with the inclusion of an intertrial interval, whereas a local analysis predicted no change in preference. Experiment 1 examined sensitivity to intertrial intervals when each was signaled by the same houselight that operated throughout the session. In Experiment 2, the intertrial interval always was signaled by the stimulus correlated with the richer terminal link. In Experiment 3, the intertrial interval was signaled by the keylights correlated with the initial links and two novel houselights. Experiment 4 provided free food pseudorandomly during the intertrial interval. In all experiments, subjects' preferences were consistent with a local analysis of choice in concurrent chains. These results are discussed in terms of delay-reduction theory, which traditionally has failed to distinguish global and local contexts.     

Effects of reinforcement context on choice

Two experiments investigated the effects of successive reinforcement contexts on choice. In the first, concurrent variable-interval schedules of primary reinforcement operated during the initial links of concurrent chains. The rate of this reinforcement arranged by the concurrent schedules was decreased across conditions: When it was higher than the terminal-link rate, preference for the higher frequency initial-link schedule increased relative to baseline. (During baseline, a standard concurrent-schedule procedure was in effect). When the initial-link reinforcement rate was lower than the terminal-link rate, preference converged toward indifference. In the second experiment, a chain schedule was available on a third key while a concurrent schedule was in effect on the side keys. When the terminal link of the chain schedule was produced, the side keys became inoperative. Availability of the chain schedule did not affect choice between the concurrent schedules. These results show that only when successive reinforcement contexts are produced by choice responding do those successive contexts affect choice in concurrent schedules.     

A FURTHER ANALYSIS OF THE VALUE OF CHOICE: CONTROLLING FOR ILLUSORY DISCRIMINATIVE STIMULI AND EVALUATING THE EFFECTS OF LESS PREFERRED ITEMS

We sought to address limitations of prior research that has isolated choice as an independent variable. Children's preferences for the opportunity to choose were evaluated in a concurrent‐chains arrangement in which identical consequences were available in choice and no‐choice conditions. Results demonstrated that preference for choice, in and of itself, was (a) evident in children, (b) not controlled by illusory discriminative stimuli such as the amount from which to choose, and (c) generally unaffected by less preferred and potentially unimportant consequences.     

Preference for free or forced choice in Sumatran orangutans (Pongo abelii)

Empirical investigations of humans, pigeons, rats, and monkeys have indicated that these species will select free over forced choice, even when faced with identical outcomes. However, the same has yet to be quantitatively confirmed in nonhuman great apes. This experiment is the first systematic investigation of preference for free or forced choice in great apes using a paradigm in which extraneous variables are highly controlled. Three orangutans were given a choice of one of two virtual routes, one that provided a choice and one that did not via a touchscreen computer program. Choice of either route was rewarded with the same type and quantity of food. Initial results indicated a preference for free choice across all three participants. However, in two control conditions, orangutans' preferences varied, suggesting a weaker tendency to exercise choice than species previously tested. We suggest further investigation of preference for free and forced choice in orangutans and other great apes through alternative experimental paradigms that focus on increasing the fidelity of free and forced choice options.     

Confidence in Personal Preferences

The first aim of this study was to test the self‐consistency model (SCM) of subjective confidence as it applies to personal preferences. According to SCM, participants presented with a two‐alternative forced‐choice (2AFC) item draw a small sample of representations of the item. Their confidence reflects the extent to which the choice is representative of the population of representations associated with the item, and the likelihood of making that choice on subsequent occasions. The second aim was to use confidence judgment as a clue to the dynamics of online preference construction. Participants were presented with 2AFC items measuring everyday personal preferences. The task was presented five times. In line with SCM, (i) when participants changed their preferences across presentations, they were systematically more confident when they made their more frequent choice; (ii) confidence in a choice in the item's first presentation predicted the likelihood of repeating that choice in subsequent presentations; (iii) despite the idiosyncratic nature of personal preferences, confidence was higher for consensual than for nonconsensual preferences; (iv) when participants predicted the preferences of others, they were also more confident when their predictions agreed with those of others; and (v) the confidence/accuracy correlation for predictions was positive for consensually correct but negative for consensually wrong predictions. These results suggest that confidence in preferences can help separate between the stable and variable contributions to preference construction in terms of the population of representations available in memory and the representations that are accessible at the time of preference solicitation, respectively. Copyright © 2012 John Wiley & Sons, Ltd.     

SOME EFFECTS OF PROCEDURAL VARIATIONS ON CHOICE RESPONDING IN CONCURRENT CHAINS

The present research used pigeons in a three‐key operant chamber and varied procedural features pertaining to both initial and terminal links of concurrent chains. The initial links randomly alternated on the side keys during a session, while the terminal links always appeared on the center key. Both equal and unequal initial‐link schedules were employed, with either differential or nondifferential terminal‐link stimuli across conditions. The research was designed to neutralize initial‐ and terminal‐link spatial cues in order to gain a clearer understanding of the roles of conditioned reinforcement and delayed primary reinforcement in choice. With both equal and unequal initial links and with differential terminal‐link stimuli, all pigeons reliably preferred the chain with the shorter terminal link. However, with equal initial links and nondifferential stimuli, all pigeons were indifferent. With unequal initial links and nondifferential stimuli, some pigeons were also indifferent, while others actually reversed and preferred the chain with the shorter initial link, even though it was followed by the longer terminal link. The decrease if not reversal of the previous preferences implies that preferences in concurrent chains are a function of the conditioned reinforcement afforded by terminal‐link stimuli, rather than delayed primary reinforcement.     

Analysis of activity preferences as a function of differential consequences

Individuals who do not possess the verbal skills to express meaningful choice in the absence of its consequences may have difficulty indicating their preference for protracted activities that are unavailable until some time in the future (e.g., taking a walk, riding a bike). When we examined the preferences of 4 individuals with developmental disabilities by showing them pictorial representations of various activities, their initial choices showed no clear preferences. In a subsequent condition, selecting a photograph resulted in brief access to the depicted activity. When selections produced differential consequences (i.e., access to the activity), clear preferences emerged. In addition, 3 individuals' preferences were later shifted to an initially less preferred but more socially desirable option by superimposing additional reinforcement contingencies for engaging in the less preferred activity. Results are discussed in terms of the conditions under which choice functions as an indicator of preference and how those conditions may be altered to improve the quality of choice making without limiting access to preferred options.     

The paradox of preference for unreliable reinforcement: The role of context and conditioned reinforcement

We discuss Belke and Spetch's (1994) work on choice between reliable and unreliable reinforcement. The studies by Belke and Spetch extend a line of basic research demonstrating that under certain experimental conditions in a concurrent chains procedure, pigeons prefer an alternative that produces unreliable reinforcement. The authors describe the variables that influence preference for unreliable reinforcement, including the signaling and the duration of the reinforcement schedules, the context in which the signaling stimuli occur, and the effects of conditioned reinforcement. Hypothetical applied examples that address these variables are provided, and their influence on preference for unreliable reinforcement in humans is discussed. We conclude by suggesting a line of applied research to examine the relationship between these variables and a preference for unreliable reinforcement.     

Some effects of interreinforcement time upon choice

Pigeons' responses were reinforced on two identical and concurrently available chain variable-interval-schedules. Unlike the typical concurrent chains procedure, both links were operative throughout, thus producing three types of concurrency: (1) concurrent initial links; (2) concurrent initial and terminal links; (3) concurrent terminal links. Choice proportions in each of these three states suggested that the pigeons were sensitive to momentary likelihoods of reinforcement: choice proportions for a schedule were higher when the schedule had been operative for some time, resulting in a higher probability of reinforcement. The study also showed that the relative rates of responding did not match the relative rates of reinforcement in any of the three states of concurrency. Instead, the choice proportions in both the concurrent initial and in the concurrent terminal links were intermediate between the scheduled and the obtained relative rates of reinforcement, while the choice proportions for a terminal link concurrent with an initial link consistently overmatched the relative interreinforcement times (and were typically 1.00). These data indicate that an accurate characterization of choice may not be obtained by considering only the relative interreinforcement interval where one interreinforcement interval is segmented into a chain. Instead, the organism's choice for a schedule will be substantially lowered by the chaining operation.     

Preference for mixed-interval versus fixed-interval schedules: number of component intervals

Six pigeons were trained under a concurrent chains procedure so that preference for fixed-interval versus mixed-interval schedules with varying numbers of component intervals could be examined. The smallest and largest intervals in the terminal links were the same value as those used by Davison (1969). Relative choice in all cases approximated the relative means of the squares of the harmonic intervals to reinforcement in the terminal links, and no effect of number of component intervals was demonstrated. Mixed-interval versus fixed-interval choice could not be predicted from extant data on fixed-interval versus fixed-interval choice.     

Dual effects on choice of conditioned reinforcement frequency and conditioned reinforcement value

Pigeons were presented with a concurrent‐chains schedule in which the total time to primary reinforcement was equated for the two alternatives (VI 30 s VI 60 s vs. VI 60 s VI 30 s). In one set of conditions, the terminal links were signaled by the same stimulus, and in another set of conditions they were signaled by different stimuli. Choice was in favor of the shorter terminal link when the terminal links were differentially signaled but in favor of the shorter initial link (and longer terminal link) when the terminal links shared the same stimulus. Preference reversed regularly with reversals of the stimulus condition and was unrelated to the discrimination between the two terminal links during the nondifferential stimulus condition. The present results suggest that the relative value of the terminal‐link stimuli and the relative rate of conditioned reinforcer presentation are important influences on choice behavior, and that models of conditioned reinforcement need to include both factors.