Short-term gains, long-term pains: How cues about state aid learning in dynamic environments

Successful investors seeking returns, animals foraging for food, and pilots controlling aircraft all must take into account how their current decisions will impact their future standing. One challenge facing decision makers is that options that appear attractive in the short-term may not turn out best in the long run. In this paper, we explore human learning in a dynamic decision making task which places short- and long-term rewards in conflict. Our goal in these studies was to evaluate how people’s mental representation of a task affects their ability to discover an optimal decision strategy. We find that perceptual cues that readily align with the underlying state of the task environment help people overcome the impulsive appeal of short-term rewards. Our experimental manipulations, predictions, and analyses are motivated by current work in reinforcement learning which details how learners value delayed outcomes in sequential tasks and the importance that “state” identification plays in effective learning.     

Navigating through abstract decision spaces: Evaluating the role of state generalization in a dynamic decision-making task

Research on dynamic decision-making tasks, in which the payoffs associated with each choice vary with participants’ recent choice history, shows that humans have difficulty making long-term optimal choices in the presence of attractive immediate rewards. However, a number of recent studies have shown that simple cues providing information about the underlying state of the task environment may facilitate optimal responding. In this study, we examined the mechanism by which this state knowledge influences choice behavior. We examined the possibility that participants use state information in conjunction with changing payoffs to extrapolate payoffs in future states. We found support for this hypothesis in an experiment in which generalizations based on this state information worked to the benefit or detriment of task performance, depending on the task’s payoff structure.     

On the cued activation of situational motivation

We examined the hypothesis that situational (or state) motivation can generalize from one situation to another via activation by associated cues. In an experimental setting, a neutral cue (a computer tone sequence) was paired repeatedly with controlling feedback. We then assessed the effect of presenting this conditioned cue during a subsequent task on participants’ motivation for that novel task. In two studies we found evidence that cued activation of controlledness significantly undermined participants’ self-determined motivation toward this subsequent task. These findings demonstrate that subtle cues, including contextual primes, can influence people’s motivational state.     

Practice variability promotes an external focus of attention and enhances motor skill learning

Variability in practice has been shown to enhance motor skill learning. Benefits of practice variability have been attributed to motor schema formation (variable versus constant practice), or more effortful information processing (random versus blocked practice). We hypothesized that, among other mechanisms, greater practice variability might promote an external focus of attention on the intended movement effect, while less variability would be more conducive to a less effective internal focus on body movements. In Experiment 1, the learning of a throwing task was enhanced by variable versus constant practice, and variable group participants reported focusing more on the distance to the target (external focus), while constant group participants focused more on their posture (internal focus). In Experiment 2, golf putting was learned more effectively with a random compared with a blocked practice schedule. Furthermore, random group learners reported using a more effective distal external focus (i.e., distance to the target) to a greater extent, whereas blocked group participants used a less effective proximal focus (i.e., putter) more often. While attentional focus was assessed through questionnaires in the first two experiments, learners in Experiment 3 were asked to report their current attentional focus at any time during practice. Again, the learning of a throwing task was more effective after random relative to blocked practice. Also, random practice learners reported using more external focus cues, while in blocked practice participants used more internal focus cues. The findings suggest that the attentional foci induced by different practice schedules might be at least partially responsible for the learning differences.     

Incidental learning,attention, and curiosity in squirrel monkeys calorie deprived as infants

The long-term effects of infantile malnutrition on information processing were studied in squirrel monkeys. From 2 through 8 weeks of age, four low-calorie infants were fed reduced amounts of a high protein diet, limiting their mean body weight gain to 23% of that of four well-fed controls, with all of this gain occurring during the first 2 weeks on the low-calorie diet. Thereafter, both groups received the standard diet ad libitum. At 68 weeks of age, incidental learning was tested by introducing shape cues that were redundant to the solution of a previously learned, two-choice, color discrimination. Removal of the color cues showed that the low-calorie monkeys had learned significantly less about the shape cues than had the controls. The basis for this incidental learning failure was then examined with an embedded-figures dicrimination, to measure attentional ability, and a novel stimulus substitution task, to measure curiosity. The groups did not differ in detecting embedded figures, but the low-calorie monkeys were significantly less likely than controls to select or handle novel stimuli that had been substituted for previously learned discriminative cues. It was concluded that undernutrition in infancy may induce a long-term strategy of only learning information which leads to an immediate reward, while suppressing the acquisition of information due to intrinsic curiosity.     

Interoceptive awareness and unaware fear conditioning: are subliminal conditioning effects influenced by the manipulation of visceral self-perception?

Research has shown repeatedly that attention influences implicit learning effects. In a similar vein, interoceptive awareness might be involved in unaware fear conditioning: The fact that the CS is repeatedly presented in the context of aversive bodily experiences might facilitate the development of conditioned responding. We investigated the role of interoceptive attention in a subliminal conditioning paradigm. Conditioning was embedded in a spatial cueing task with subliminally presented cues that were followed by a masking stimulus. Response times to the targets that were either validly or invalidly predicted by the cues served as index of conditioning. Interoceptive attention was manipulated between-subjects. Half the participants completed a heartbeat detection task before conditioning. This task tunes attention to one’s own bodily signals. We found that conditioned responding was facilitated in this latter group of participants. These results are in line with the hypothesis that a rise interoceptive attention enhances unaware conditioned responding.     

The absence of numbers to express the amount may affect delay discounting with humans

Human delay discounting is usually studied with experimental protocols that use symbols to express delay and amount. In order to further understand discounting, we evaluated whether the absence of numbers to represent reward amounts affects discount rate in general, and whether the magnitude effect is generalized to nonsymbolic situations in particular. In Experiment 1, human participants were exposed to a delay‐discounting task in which rewards were presented using dots to represent monetary rewards (nonsymbolic); under this condition the magnitude effect did not occur. Nevertheless, the magnitude effect was observed when equivalent reward amounts were presented using numbers (symbolic). Moreover, in estimation tasks, magnitude increments produced underestimation of large amounts. In Experiment 2, participants were exposed only to the nonsymbolic discounting task and were required to estimate reward amounts in each trial. Consistent with Experiment 1, the absence of numbers representing reward amounts produced similar discount rates of small and large rewards. These results suggest that value of nonsymbolic rewards is a nonlinear function of amount and that value attribution depends on perceived difference between the immediate and the delayed nonsymbolic rewards.     

Fourteen-month-old infants copy an action style accompanied by social-emotional cues

Human adults exaggerate their actions and facial expressions when interacting with infants. These infant-directed modifications highlight certain aspects of action sequences and attract infants’ attention. This study investigated whether social-emotional aspects of infant-directed modifications, such as smiling, eye contact, and onomatopoeic vocalization, influence infants’ copying of another's action, especially action style, during the process of achieving an outcome. In Study 1, 14-month-old infants (n = 22) saw an experimenter demonstrate goal-directed actions in an exaggerated manner. Either the style or the end state of the actions was accompanied by social-emotional cues from the experimenter. Infants copied the style of the action more often when social-emotional cues accompanied the style than when they accompanied the end state. In Study 2, a different group of 14-month-old infants (n = 22) watched the same exaggerated actions as in Study 1, except that either the style or the end state was accompanied by a physical sound instead of social-emotional cues. The infants copied the end state consistently more often than the style. Taken together, these two studies showed that accompanying social-emotional cues provided by a demonstrator, but not accompanying physical sound, increased infants’ copying of action style. These findings suggest that social-emotional cues facilitate efficient social learning through the adult–infant interaction.     

Children's theories of motivation

To investigate children’s theories of motivation, we asked 166 children (8-12 years of age) to rate the effect of various motivational strategies on task interest, over the short and long terms, in activities described as appealing or unappealing. Children viewed the rewards strategy as resulting in greatest interest except when implemented over the long term for appealing activities. Individual difference analyses revealed that some children held operant theories of motivation, in which rewards were central, and others held hybrid theories, in which rewards were key, but some allowance was made for interest to be self-sustaining in the absence of inducements. Children’s theories predicted their academic self-regulation. Their theories are discussed relative to an expert theory of motivation.     

Using stop signals to inhibit chronic dieters' responses toward palatable foods

Palatable foods in the environment can unintentionally trigger reactions to obtain them, which may interfere with dieting attempts. We tested a strategy to facilitate dieting behavior that makes use of behavioral stop signals that should instantly inhibit chronic dieters’ responses toward palatable foods. Participants performed a go/no-go task in which go cues and no-go cues (i.e., the behavioral stop signals) were presented with pictures of palatable foods and control objects. In Study 1, we tested the immediate behavioral effect of presenting stop signals near palatable foods in a reaction time paradigm. In Study 2 we assessed consumption of palatable food that had either consistently been associated with no-go cues, or not. Results show that no-go cues instantly inhibited responses toward palatable foods especially among chronic dieters. Moreover, across a one day period chronic dieters consumed less of a food that had consistently been associated with no-go cues. Stop signals thus appear a promising tool for chronic dieters to control behavior to palatable foods, and we discuss the merits and potential applications of this tool for facilitating dieting behavior.     

Do narcissists make better decisions? An investigation of narcissism and dynamic decision-making performance

We investigated whether narcissism affected dynamic decision-making performance in the presence and absence of misleading information. Performance was examined in a two-choice dynamic decision-making task where the optimal strategy was to forego an option providing larger immediate rewards in favor of an option that led to larger delayed rewards. Information regarding foregone rewards from the alternate option was presented or withheld to bias participants toward the sub-optimal choice. The results demonstrated that individuals high in narcissistic traits performed comparably to low narcissism individuals when foregone reward information was absent, but high narcissism individuals outperformed individuals low in narcissistic traits when misleading information was presented. The advantage for participants high in narcissistic traits was strongest within males, and, overall, males outperformed females when foregone rewards were present. While prior research emphasizes narcissists’ decision-making deficits, our findings provide evidence that individuals high in narcissistic traits excel at decision-making tasks that involve disregarding ambiguous information and focusing on the long-term utility of each option. Their superior ability at filtering out misleading information may reflect an effort to maintain their self-view or avoid ego threat.     

Money enhances memory consolidation--but only for boring material

Money’s ability to enhance memory has received increased attention in recent research. However, previous studies have not directly addressed the time-dependent nature of monetary effects on memory, which are suggested to exist by research in cognitive neuroscience, and the possible detrimental effects of monetary rewards on learning interesting material, as indicated by studies in motivational psychology. By utilizing a trivia question paradigm, the current study incorporated these perspectives and examined the effect of monetary rewards on immediate and delayed memory performance for answers to uninteresting and interesting questions. Results showed that monetary rewards promote memory performance only after a delay. In addition, the memory enhancement effect of monetary rewards was only observed for uninteresting questions. These results are consistent with both the hippocampus-dependent memory consolidation model of reward learning and previous findings documenting the ineffectiveness of monetary rewards on tasks that have intrinsic value.     

Choice of practice-task order enhances golf skill learning

ObjectiveThe purpose of the present study was to examine whether supporting learners’ autonomy, by giving them a small choice (i.e., order of practice devices) while practicing a golf putting task, would enhance learning, confidence, and positive affect.DesignExperimental, between-participants, and yoked design.MethodsTwo groups of participants practiced a golf-putting task under choice or control conditions. Choice group participants selected the order of three practice devices (visual cues, auditory cues, chest bar), while control group participants had to use those devices in the same order as their yoked choice-group counterpart. Learning was assessed by a delayed retention test. In addition to putting accuracy, we measured learners’ perceived choice, confidence, and positive affect.ResultsPractice and retention performance were enhanced in the choice relative to the control group. Perceived choice, confidence, and positive affect were rated higher by choice group participants as well.ConclusionsProviding performers with a small choice during task practice had motivational benefits that resulted in enhanced learning, increased confidence, and more positive emotional responses.     

When and how less is more: reply to Tharp and Pickering

In DeCaro et al. [DeCaro, M. S., Thomas, R. D., & Beilock, S. L. (2008). Individual differences in category learning: Sometimes less working memory capacity is better than more. Cognition, 107, 284-294] we demonstrated that sometimes less working memory (WM) has its advantages. The lower individuals’ WM, the faster they achieved success on an information-integration (II) category learning task adopted from Waldron and Ashby [Waldron, E. M., & Ashby, F. G. (2001). The effects of concurrent task interference on category learning: Evidence for multiple category learning systems. Psychonomic Bulletin & Review, 8, 168-176]. We attributed this success to the inability of lower WM individuals to employ explicit learning strategies heavily reliant on executive control. This in turn, we hypothesized, might push lower WM individuals to readily adopt procedural-based strategies thought to lead to success on the II task. Tharp and Pickering [Tharp, I. J., & Pickering, A. D. (2009). A note on DeCaro, Thomas, and Beilock (2008): Further data demonstrate complexities in the assessment of information-integration category learning. Cognition] recently questioned whether the II category learning task DeCaro et al. used really reflects procedural learning. In an effort to investigate Tharp and Pickering’s assertions with respect to individual differences in WM, we replicate and extend our previous work, in part by modeling participants’ response strategies during learning. We once again reveal that lower WM individuals demonstrate earlier II learning than their higher WM counterparts. However, we also show that low WM individuals’ initial success is not because of procedural-based responding. Instead, individuals lower in WM capacity perseverate in using simple rule-based strategies that circumvent heavy demands on WM while producing above-chance accuracy.     

Unique behavioral strategies in visuomotor learning: Hope for the non-learner

The existence of individual differences in motor learning capability is well known but the behaviors or strategies that contribute to this variability have been vastly understudied. What performance characteristics distinguish an expert level performer from individuals who experience little to no success, those labeled non-learners? We designed a rule-based visuomotor task which requires identification (discovery) and then exploitation of specific explicit and implicit task components that requires a specific movement pattern, the task rule, for goal achievement. When participants first attempt the task, they are informed about the goal, but are naïve to the task rule. Therefore, the purpose of this experiment is to determine how acquisition of both implicit and explicit task components, the inherent elements of the task rule, reveals differing strategies associated with performance and task success. We test the hypothesis that an examination of performance will reveal sub-groups with varying levels of success. Further, for each subgroup, we expect to find a unique relationship between visual Time-in-Target feedback (a measure of success) and subsequent updating of each task component. Out of 32 non-disabled adults, we identified three distinct sub-groups: (Low Performer/Non-Learner (LP, N = 9), Moderate Performer (MP, N = 12) and High Performer (HP, N = 11)). A quantitative analysis of behavioral patterns reveals three findings: First, the LP sub-group demonstrated significantly lower task success which was associated with difficulty identifying the explicit component of the task. Second, the HP sub-group acquired the two task components in parallel over practice. Third, when both explicit and implicit component performance is plotted across sub-groups, a task component continuum emerges that seamlessly progresses from low to moderate to high performer groups. An exploratory analysis reveals that self-reported level of prior lifetime accumulation of video game and physical activity experience is a significant predictor of individual task performance (R² = 0.50). In summary, what appears to be a key distinction between varying levels of human rule-based motor learning is the process by which feedback is used to update performance of inherent elements of the task rule. Evidence of a performance continuum and limited prior experience suggests that Low Performer/Non-Learners are generally inexperienced with these kinds of tasks, although the role of genetics and other innate learning capabilities in visuomotor learning is still largely unknown. These findings provoke new research directions toward probing the differential performance strategies associated with expertise and the development of interventions aimed to convert non-learners into learners.     

Successive and conditional discrimination learning in pigs

We studied the ability of pigs to discriminate tone cues using successive and conditional discrimination tasks. Pigs (n = 8) were trained in a successive discrimination Go/No-Go task (Experiment 1) to associate a Go-cue with a reward at the end of a runway and a No-Go-cue with the absence of reward. Latency to reach the goal-box was recorded for each cue-type. Learning of a conditional discrimination task was compared between low-birthweight (LBW, n = 5) and normal-birthweight (NBW, n = 6) pigs (Experiment 2) and between conventional farm (n = 7) and Göttingen miniature (n = 8) pigs (Experiment 3). In this active-choice task, one cue signalled a response in a right goal-box was correct and a second cue signalled a response in a left goal-box was correct. Cues were differentially rewarded. The number of sessions to learn the discrimination and number of correct choices per cue-type were recorded. In Experiment 1, four out of eight pigs showed learning on the task, that is, a higher latency to respond to the No-Go-cue, within 25 sessions. In Experiment 2, eight out of 11 pigs learned the discrimination within 46 sessions. LBW learners learned faster than NBW learners. In Experiment 3, all 15 pigs learned the task within 16 sessions. Göttingen miniature pigs learned faster than conventional farm pigs. While some methodological issues may improve the Go/No-Go design, it is suggested that an active-choice task yields clearer and more consistent results than one relying on latency alone.     

Inferring Behavior From Partial Social Information Plays Little or No Role in the Cultural Transmission of Adaptive Traits

Many human cultural traits become increasingly beneficial as they are repeatedly transmitted, thanks to an accumulation of modifications made by successive generations. But how do later generations typically avoid modifications which revert traits to less beneficial forms already sampled and rejected by earlier generations? And how can later generations do so without direct exposure to their predecessors' behavior? One possibility is that learners are sensitive to cues of non-random production in others' behavior, and that particular variants (e.g., those containing structural regularities unlikely to occur spontaneously) have been produced deliberately and with some effort. If this non-random behavior is attributed to an informed strategy, then the learner may infer that apparent avoidance of certain possibilities indicates that these have already been sampled and rejected. This could potentially prevent performance plateaus resulting from learners modifying inherited behaviors randomly. We test this hypothesis in four experiments in which participants, either individually or in interacting dyads, attempt to locate rewards in a search grid, guided by partial information about another individual's experience of the task. We find that in some contexts, valid inferences about another's behavior can be made from partial information, and these inferences can be used in a way which facilitates trait adaptation. However, the benefit of these inferences appears to be limited, and in many contexts—including some which have the potential to make inferring the experience of another individual easier—there appears to be no benefit at all. We suggest that inferring previous behavior from partial social information plays a minimal role in the adaptation of cultural traits.     

Using variability to guide dimensional weighting: associative mechanisms in early word learning

At 14 months, children appear to struggle to apply their fairly well-developed speech perception abilities to learning similar sounding words (e.g., bih/dih; Stager & Werker, 1997). However, variability in nonphonetic aspects of the training stimuli seems to aid word learning at this age. Extant theories of early word learning cannot account for this benefit of variability. We offer a simple explanation for this range of effects based on associative learning. Simulations suggest that if infants encode both noncontrastive information (e.g., cues to speaker voice) and meaningful linguistic cues (e.g., place of articulation or voicing), then associative learning mechanisms predict these variability effects in early word learning. Crucially, this means that despite the importance of task variables in predicting performance, this body of work shows that phonological categories are still developing at this age, and that the structure of noninformative cues has critical influences on word learning abilities.     

Pigeons respond to produce periods in which rewards are independent of responding

Pecks by pigeons on a response key produced an ON state during which intermittent rewards were freely available, i.e., independently of responding. Pecks during the ON state caused it to remain ON. If no pecks occurred, the state changed to OFF—the key color changed—and rewards were not presented. The state remained OFF until the next response. Thus, responses controlled the state in the chamber but did not cause immediate reinforcement. Four dimensions of the schedule were varied: the rates of response-independent rewards during ON; the duration of ON produced by each peck; the pattern of rewards during ON; and the presence vs absence of exteroceptive cues during ON and OFF. The results showed that rates of responding were primarily controlled by the duration of ON produced by each response. When each response caused a long period of ON, pecks occurred infrequently; when each response caused a brief period of ON, pecks were frequent.