Probability matching in risky choice: The interplay of feedback and strategy availability

Probability matching in sequential decision making is a striking violation of rational choice that has been observed in hundreds of experiments. Recent studies have demonstrated that matching persists even in described tasks in which all the information required for identifying a superior alternative strategy—maximizing—is present before the first choice is made. These studies have also indicated that maximizing increases when (1)?the asymmetry in the availability of matching and maximizing strategies is reduced and (2)?normatively irrelevant outcome feedback is provided. In the two experiments reported here, we examined the joint influences of these factors, revealing that strategy availability and outcome feedback operate on different time courses. Both behavioral and modeling results showed that while availability of the maximizing strategy increases the choice of maximizing early during the task, feedback appears to act more slowly to erode misconceptions about the task and to reinforce optimal responding. The results illuminate the interplay between “top-down” identification of choice strategies and “bottom-up” discovery of those strategies via feedback.     

Probability matching in choice under uncertainty: Intuition versus deliberation

Gaissmaier and Schooler (2008) [Gaissmaier, W., & Schooler, L. J. (2008). The smart potential behind probability matching. Cognition, 109, 416-422] argue that probability matching, which has traditionally been viewed as a decision making error, may instead reflect an adaptive response to environments in which outcomes potentially follow predictable patterns. In choices involving monetary stakes, we find that probability matching persists even when it is not possible to identify or exploit outcome patterns and that many “probability matchers” rate an alternative strategy (maximizing) as superior when it is described to them. Probability matching appears to reflect a mistaken intuition that can be, but often is not, overridden by deliberate consideration of alternative choice strategies.     

Behavioral and pharmacological variables affecting risky choice in rats.

The effects of manipulations of response requirement, intertrial interval (ITI), and psychoactive drugs (ethanol, phencyclidine, and d-amphetamine) on lever choice under concurrent fixed-ratio schedules were investigated in rats. Responding on the "certain' lever produced three 45-mg pellets, whereas responding on the "risky" lever produced either 15 pellets (p = .33) or no pellets (p .67). Rats earned all food during the session, which ended after 12 forced trials and 93 choice trials or 90 min, whichever occurred first. When the response requirement was increased from 1 to 16 and the ITI was 20 s, percentage of risky choice was inversely related to fixed-ratio value. When only a single response was required but the ITI was manipulated between 20 and 120 s (with maximum session duration held constant), percentage of risky choice was directly related to length of the ITI. The effects of the drugs were investigated first at an ITI of 20 s, when risky choice was low for most rats, and then at an ITI of 80 s, when risky choice was higher for most rats. Ethanol usually decreased risky choice. Phencyclidine did not usually affect risky choice when the ITI was 20 s but decreased it in half the rats when the ITI was 80 s. For d-amphetamine, the effects appeared to he related to baseline probability of risky choice; that is, low probabilities were increased and high probabilities were decreased. Although increase in risky choice as a function of the ITI is at variance with previous ITI data, it is consistent with foraging data showing that risk aversion decreases as food availability decreases. The pharmacological manipulations showed that drug effects on risky choice may be influenced by the baseline probability of risky choice, just as drug effects can be a function of baseline response rate.     

Probability matching: encouraging optimal responding in humans.

Two hundred undergraduate students participated in a repeated-trials binary choice procedure in which choice of one outcome was correct on 75% of trials. Subjects received 192 trials and were divided into five conditions: (1) control; (2) subjects were given the actual probabilities; (3) subjects were told if they did well they could leave early; (4) competition condition; (5) midway through the task subjects were asked to recommend a strategy for another subject. Half of the subjects in each group were told that the best they could do was to be correct on 75% of the trials. This manipulation permitted assessment of the hypothesis that subjects in probability-matching tasks are seeking a strategy that will be correct on 100% of the trials. The results partially confirmed this hypothesis. In addition, two of the variables improved performance significantly (giving probabilities and asking subjects to recommend a strategy). However, while subjects in all groups improved significantly over trials, optimal choice did not occur in this task.     

When confidence in a choice is independent of which choice is made

For forced-choice two-alternative general-information questions, confidence in the correctness of the answer differed reliably for different questions, regardless of which answer was chosen. Results suggested that this choice-independent confidence is mediated by the domain familiarity of the question and by its tendency to bring to mind either few or many thoughts and considerations. Ratings of the questions on familiarity and accessibility yielded strong correlations with participants’ confidence in whichever of two answers they had chosen, and with estimates of the percentage of participants who were likely to have chosen either of the answers in a previous experiment. The results were interpreted in terms of confirmation bias: Because items differ in the extent to which they bring to mind few or many pertinent thoughts, selective focusing on supportive evidence should yield a positive correlation between mean confidence in one answer and mean confidence in the alternative answer, as if there is no competition between them.     

Goal attainment as a resource: The cushion effect in risky choice above a goal

Goals are a ubiquitous part of life and have been shown to change behavior in many domains. This research studied the influence of goal attainment on risky choice behavior. Previous research has shown that goals tend to increase risk‐seeking behavior when potential outcomes fall below a goal. We examined a new problem: Choice behavior when all potential outcomes in a choice set achieve or exceed the goal. Two studies show a “cushion effect” of goal attainment on choice under risk. When all possible outcomes of all options are above a salient and specific goal, decision makers are more likely to choose a risky option over a certain outcome with equal expected value (EV). We hypothesized that the attainment of a goal serves as a cushion that softens the negative emotions associated with receiving a gamble's low outcome. This allows risk taking that would otherwise be unattractive. Copyright © 2009 John Wiley & Sons, Ltd.     

Probability and delay of reinforcement as factors in discrete-trial choice.

Pigeons chose between two alternatives that differed in the probability of reinforcement and the delay to reinforcement. A peck on the red key always produced a delay of 5 s and then a possible reinforcer. The probability of reinforcement for responding on this key varied from .05 to 1.0 in different conditions. A response on the green key produced a delay of adjustable duration and then a possible reinforcer, with the probability of reinforcement ranging from .25 to 1.0 in different conditions. The green-key delay was increased or decreased many times per session, depending on a subject's previous choices. The purpose of these adjustments was to estimate an indifference point, or a delay that resulted in a subject's choosing each alternative about equally often. In conditions where the probability of reinforcement was five times higher on the green key, the green-key delay averaged about 12 s at the indifference point. In conditions where the probability of reinforcement was twice as high on the green key, the green-key delay at the indifference point was about 8 s with high probabilities and about 6 s with low probabilities. An analysis based on these results and those from studies on delay of reinforcement suggests that pigeons' choices are relatively insensitive to variations in the probability of reinforcement between .2 and 1.0, but quite sensitive to variations in probability between .2 and 0.     

Signal detection and matching: analyzing choice on concurrent variable-interval schedules.

Pigeons' pecks on a red key and a green key were followed by access to grain according to pairs of concurrent independent variable-interval schedules in a combined signal detection/matching law paradigm. Pecks on the red key were reinforced by the richer variable-interval schedule if a short-duration tone had been presented; pecks on the green key were reinforced by the richer variable-interval schedule if a long-duration tone had been presented. Pecks on the green key given a short-duration tone, or on the red key given a long-duration tone, were reinforced by the leaner variable-interval schedule. The data were analyzed according to both signal detection's and the matching law's separate measures of, first, the discrimination of the choices and, second, the bias to make one response or another. Increasing the difficulty of the tone-duration discrimination decreased both methods' measures of the discrimination of the choices and did not change both methods' measures of the bias to make one response or another. Changing the leaner variable-interval schedule so that it approached the richer variable-interval schedule decreased signal detection's measure of discrimination but left its measure of response bias and the matching law measures unchanged. Data collected only until a subject's first changeover response following presentation of a long or a short tone showed higher values for both methods' measures of discrimination, no change in signal detection's measure of response bias, and lower values for the matching law's measure of response bias. Relationships between the matching law's and signal detection's methods of analyzing choice are discussed. It is concluded that a signal detection analysis is more efficient for examining changes in the difficulty of a discrimination, whereas a matching law analysis is more effective for examining the effects of changes in relative reinforcer frequency.     

Delayed reinforcement and delayed choice in symbolic matching to sample: Effects on stimulus discriminability

Six pigeons were trained to peck a red side key when the brighter of two white lights (S₁) had been presented on the center key, and to peck a green side key when the dimmer of two white lights (S₂) had been presented on the center key. Equal frequencies of reinforcers were provided for the two types of correct choice. Incorrect choices, red side-key pecks following S₂ presentations and green side-key pecks following S₁ presentations, resulted in blackout. With 0-s delay between choice and reinforcement, the delay between sample presentation and choice was varied from 0 to 20 s. Then, with 0-s delay between sample presentation and choice, the delay between choice and reinforcement was varied from 0 to 20 s. Both types of delay resulted in decreased discriminability (defined in terms of a signal-detection analysis) of the center-key stimuli, but delayed choice had more effect on discriminability than did delayed reinforcement. These data are consistent with the view that the two kinds of delay operate differently. The effect of a sample-choice delay may result from a degradation of the conditional discriminative stimuli during the delay; the effect of a choice-reinforcer delay may result from a decrement in control by differential reinforcement.     

Seeing is not enough: manipulating choice options causes focusing and preference change in multiattribute risky decision‐making

We show that preferences depend on the attributes that can be directly manipulated when people need to integrate multiple sources of information because direct manipulation causes focusing bias. This effect appears even when all relevant information is simultaneously and explicitly presented at the time the decisions are made. Participants decided how much to save, what investment risk to take and observed the future financial consequences in terms of the mean and variability of the expected retirement income. Participants who manipulated only the future income distribution saved more and took less risk. This effect disappears when the risk‐related variables are removed, which indicates that task complexity is a mediator of such focusing effects. A more balanced trade‐off between the choice attributes was selected when all attributes were manipulated. However, when there is a dichotomy between manipulating versus observing choice attributes, then decisions were based mostly on the manipulated attributes. Copyright © 2008 John Wiley & Sons, Ltd.     

Beyond gains and losses: the effect of need on risky choice in framed decisions

Substantial evidence suggests people are risk-averse when making decisions described in terms of gains and risk-prone when making decisions described in terms of losses, a phenomenon known as the framing effect. Little research, however, has examined whether framing effects are a product of normative risk-sensitive cognitive processes. In 5 experiments, it is demonstrated that framing effects in the Asian disease problem can be explained by risk-sensitivity theory, which predicts that decision makers adjust risk acceptance on the basis of minimal acceptable thresholds, or need. Both explicit and self-determined need requirements eliminated framing effects and affected risk acceptance consistent with risk-sensitivity theory. Furthermore, negative language choice in loss frames conferred the perception of high need and led to the construction of higher minimal acceptable thresholds. The results of this study suggest that risk-sensitivity theory provides a normative rationale for framing effects based on sensitivity to minimal acceptable thresholds, or needs.     

The general matching law describes choice on concurrent variable-interval schedules of wheel-running reinforcement.

Six male Wistar rats were exposed to concurrent variable-interval schedules of wheel-running reinforcement. The reinforcer associated with each alternative was the opportunity to run for 15 s, and the duration of the changeover delay was 1 s. Results suggested that time allocation was more sensitive to relative reinforcement rate than was response allocation. For time allocation, the mean slopes and intercepts were 0.82 and 0.008, respectively. In contrast, for response allocation, mean slopes and intercepts were 0.60 and 0.03, respectively. Correction for low response rates and high rates of changing over, however, increased slopes for response allocation to about equal those for time allocation. The results of the present study suggest that the two-operant form of the matching law can be extended to wheel-running reinforcement. 'I'he effects of a low overall response rate, a short Changeover delay, and long postreinforcement pausing on the assessment of matching in the present study are discussed.     

Dolphin pointing is linked to the attentional behavior of a receiver

In 2001, Xitco et al. (Anim Cogn 4:115–123) described spontaneous behaviors in two bottlenose dolphins (Tursiops truncatus) that resembled pointing and gaze alternation. The dolphins spontaneous behavior was influenced by the presence of a potential receiver, and the distance between the dolphin and the receiver. The present study adapted the technique of Call and Tomasello [(1994) J Comp Psychol 108:307–317], used with orangutans to test the effect of the receivers orientation on pointing in these same dolphins. The dolphins directed more points and monitoring behavior at receivers whose orientation was consistent with attending to the dolphins. The results demonstrated that the dolphins pointing and monitoring behavior, like that of apes and infants, was linked to the attentional behavior of the receiver.     

Stock optimizing in choice when a token deposit is the operant.

Each of 2 monkeys typically earned their daily food ration by depositing tokens in one of two slots. Tokens deposited in one slot dropped into a bin where they were kept (token kept). Deposits to a second slot dropped into a bin where they could be obtained again (token returned). In Experiment 1, a fixed-ratio (FR) 5 schedule that provided two food pellets was associated with each slot. Both monkeys preferred the token-returned slot. In Experiment 2, both subjects chose between unequal FR schedules with the token-returned slot always associated with the leaner schedule. When the FRs were 2 versus 3 and 2 versus 6, preferences were maintained for the token-returned slot; however, when the ratios were 2 versus 12, preference shifted to the token-kept slot. In Experiment 3, both monkeys chose between equal-valued concurrent variable-interval variable-interval schedules. Both monkeys preferred the slot that returned tokens. In Experiment 4, both monkeys chose between FRs that typically differed in size by a factor of 10. Both monkeys preferred the FR schedule that provided more food per trial. These data show that monkeys will choose so as to increase the number of reinforcers earned (stock optimizing) even when this preference reduces the rate of reinforcement (all reinforcers divided by session time).     

Resting gamma power is linked to reading ability in adolescents

Infants who have more power within the gamma frequency range at rest develop better language and cognitive abilities over their first 3 years of life (Benasich et al., 2008). This positive trend may reflect the gradual increase in resting gamma power that peaks at about 4 years (Takano & Ogawa, 1998): infants further along the maturational curve may exhibit both increased resting gamma power and more advanced language and cognitive function. Similar to other neural characteristics such as synaptic density, resting gamma power subsequently decreases with further development into adulthood (Tierney, Strait, O'Connell & Kraus, 2013). If previously reported relationships between resting gamma power and behavioral performance reflect variance in maturation, at least in part, negative correlations between resting gamma and behavior may predominate in later developmental stages, during which resting gamma activity is decreasing. We tested this prediction by examining resting gamma activity and language‐dependent behavioral performance, reflected by a variety of reading‐related tests, in adolescents between the ages of 14 and 15 years. Consistent with our predictions, resting gamma power inversely related to every aspect of reading assessed (i.e. reading fluency, rapid naming, and basic reading proficiency). Our results suggest that resting gamma power acts as an index of maturational progress in adolescents.     

Learning mechanisms in matching to sample.

A model system and an experiment on early learning and decision processes in matching-to-sample and oddity-from-sample tasks are presented. The model system is based, in part, on videotaped records of pigeons' looking responses before they chose 1 of 2 comparison stimuli. In order to see the wavelength stimuli recessed behind the pecking keys, the pigeons had to move in front of them. Although there were slight increases in the acceptance probability with switches between the stimuli before a choice response, the overall decision strategy was close to a Markov choice process in which choice proportions could be predicted by the product of each rejection probability and the final acceptance probability. Learning involved learning to discriminate rather than learning to adopt a stricter criterion for an acceptable sample match.     

One-sided limb preference is linked to alternating-limb locomotion in anuran amphibians

Amphibians provide a unique opportunity for identifying possible links between lateralized behaviors, locomotion, and phylogeny and for addressing the origin of lateralized behaviors of higher vertebrates. Five anuran species with different locomotive habits were tested for forelimb and hind limb preferences during 2 stereotyped behavior sequences--wiping a foreign object off their snout and righting themselves from the overturned position. The experiments were analyzed in a broader context of previous findings on anuran lateralization involving 11 anuran species that were studied within the same experimental paradigms. This analysis shows that one-sided forelimb and hind limb motor lateralization in anurans is strongly associated with alternating-limb locomotion and other unilateral limb activity. Conclusions reached for anuran amphibians may be applicable to other vertebrates possessing paired appendages-the degree of lateralization in motor response depends on the mode of locomotion used by a species.