Computational models for the combination of advice and individual learning

Decision making often takes place in social environments where other actors influence individuals' decisions. The present article examines how advice affects individual learning. Five social learning models combining advice and individual learning-four based on reinforcement learning and one on Bayesian learning-and one individual learning model are tested against each other. In two experiments, some participants received good or bad advice prior to a repeated multioption choice task. Receivers of advice adhered to the advice, so that good advice improved performance. The social learning models described the observed learning processes better than the individual learning model. Of the models tested, the best social learning model assumes that outcomes from recommended options are more positively evaluated than outcomes from nonrecommended options. This model correctly predicted that receivers first adhere to advice, then explore other options, and finally return to the recommended option. The model also predicted accurately that good advice has a stronger impact on learning than bad advice. One-time advice can have a long-lasting influence on learning by changing the subjective evaluation of outcomes of recommended options.     

Adapted to explore: Reinforcement learning in Autistic Spectrum Conditions

Recent studies have recorded a tendency of individuals with Autism Spectrum Conditions (ASC) to continually change their choices in repeated choice tasks. In the current study we examine if this finding implies that ASC individuals have a cognitive style that facilitates exploration and discovery. Six decision tasks were administered to adolescents with ASC and matched controls. Significant differences in shifting between choice options appeared in the Iowa Gambling task (Bechara, Damasio, Damasio, & Anderson, 1994). A formal cognitive modeling analysis demonstrated that for about half of the ASC participants the adaptation process did not conform to the standard reinforcement learning model. These individuals were only coarsely affected by choice-outcomes, and were more influenced by the exploratory value of choices, being attracted to previously un-explored alternatives. An examination of the five simpler decision tasks where the advantageous option was easier to determine showed no evidence of this pattern, suggesting that the shifting choice pattern is not an uncontrollable tendency independent of task outcomes. These findings suggest that ASC individuals have a unique adaptive learning style, which may be beneficial is some learning environment but maladaptive in others, particularly in social contexts.     

Correcting erroneous inferences in memory: The role of source credibility

People often continue to rely on erroneous information about people and events, even in the face of subsequent counter information. The current study examined whether this information could be effectively corrected by a credible source. We examined two aspects of credibility: trustworthiness and expertise. Experiment 1 showed that receiving a correction from a source high in trustworthiness and expertise reduced participants’ use of original information when making inferences. Experiment 2 showed that source expertise alone was not sufficient to reduce participants’ reliance on the original information. The results from Experiment 3 showed that source trustworthiness alone significantly decreased participants’ use of the original information when making inferences. The results suggest that people may be able to reduce their use of original information if they receive a correction from a person who is deemed to be highly trustworthy. These findings have implications for decision making in politics and other applied areas.     

Transfer of avoidance responding to a sensory preconditioned cue: Evidence for the role of S-S and R-S knowledge in avoidance learning

In avoidance learning studies, a warning signal (Sd) is directly paired with an aversive unconditioned stimulus (US) unless a particular response is performed. Research by Augustson and Dougher (1997) demonstrated that not only warning signals but also stimuli that are indirectly paired with the aversive event are capable of affecting the avoidance response. In the present study, we extended these results by incorporating a sensory preconditioning paradigm in an avoidance learning procedure. Sensory preconditioning training influenced the selection of avoidance responses in the presence of warning signals that were never paired directly with the aversive events. Moreover, results suggest that the selection of the avoidance responses was based on an integration of knowledge about the relation between the Sd and the US (S-S knowledge) and knowledge about the relation between the avoidance response and the US (R-S knowledge). As the expectancy-based theory of Lovibond (2006) is the only theory of avoidance that incorporates both knowledge structures, the results provide unique support for this theory.     

Reluctant to change: Self-enhancing responses to diverging performance measures

Although there is extensive evidence that past performance influences the propensity to make changes, research on how decision makers respond to diverging performance measures has been sparse. This paper addresses this gap in an experimental and a field study in which we examine how decision makers respond to the ambiguity introduced by two diverging performance indicators of unequal importance. Both studies suggest that decision makers respond to diverging performance indicators in a self-enhancing manner. Decision makers gave importance to a secondary performance indicator only when it helped them maintain a sense of positive performance, that is, when a secondary performance measure was high and a primary performance measure was low. The results suggest that, in contexts in which decision makers are likely to experience diverging performance indicators, perceptions of success and the associated reluctance to make changes might be more pervasive than is often thought.     

Short-term gains, long-term pains: How cues about state aid learning in dynamic environments

Successful investors seeking returns, animals foraging for food, and pilots controlling aircraft all must take into account how their current decisions will impact their future standing. One challenge facing decision makers is that options that appear attractive in the short-term may not turn out best in the long run. In this paper, we explore human learning in a dynamic decision making task which places short- and long-term rewards in conflict. Our goal in these studies was to evaluate how people’s mental representation of a task affects their ability to discover an optimal decision strategy. We find that perceptual cues that readily align with the underlying state of the task environment help people overcome the impulsive appeal of short-term rewards. Our experimental manipulations, predictions, and analyses are motivated by current work in reinforcement learning which details how learners value delayed outcomes in sequential tasks and the importance that “state” identification plays in effective learning.     

Dissociation of sensory facilitation and decision bias in statistical learning

ABSTRACT

The detection of regularities in the sensory environment, known as statistical learning, is an important brain function that has been observed in many experimental contexts. In these experiments, statistical learning of patterned sensory stimulation leads to improvements in the speed and/or accuracy with which subsequent stimuli are recognized. That is, statistical learning facilitates the transformation of sensory stimuli into motor responses, but the mechanism by which this occurs is unclear. Statistical learning could improve the efficiency of sensory processing, or it could bias responses toward particular outcomes. The distinction is important, as these different hypotheses imply different functions and different neural substrates for statistical learning. Here we address this problem by studying statistical learning as a decision-making process, which allows us to leverage the extensive computational literature on this topic. Specifically we describe a method for applying the Diffusion Decision Model (DDM) to isolate different sensory and cognitive processes associated with decision-making. The results indicate that statistical learning improves performance on a visual learning task in two distinct ways: by altering the efficiency of sensory processing and by introducing biases in the decision-making process. By fitting the parameters of the DDM to data from individual subjects, we find that the prominence of these two factors differed substantially across the population, and that these differences were predictive of individual performance on the psychophysical task. Overall, these results indicate that different cognitive processes can be recruited by statistical learning, and that the DDM is a powerful framework for detecting these influences.     

The ecology of self-monitoring effects on memory of verbal productions: Does speaking to someone make a difference?

Experiments involving verbal self-monitoring show that memory for spoken words varies with types of sensory feedback: memory is better when words are spoken aloud than when they are lip-synched or covertly produced. Such effects can be explained by the Central Monitoring Theory (CMT) via a process that matches a forward model reflecting expected sensory effects of practiced forms and sensory information during speech. But CMT oversees factors of shared attention as achieved by speaker–listener gaze, and implies that sensory feedback may not affect the learning of unpracticed forms (non-words). These aspects of CMT were examined in two experiments of self-monitoring focusing on oro-sensory feedback. In Experiment 1 we show that varying feedback creates differential effects on memory for spoken words and that speaker–listener gaze alters these effects. Using non-words, Experiment 2 shows the absence of differential feedback effects. The results confirm CMT but suggest the need to refine the theory in terms of processes that mediate attention.     

Opposite effects of feedback contingency on the process of risky decisions-making

The main aim of this study was to look into the effect of feedback contingency on decision making in complex risky situations, using a driving context. Participants had to decide braking or not in a set of risky traffic situations. After the response, a negative non-contingent or contingent feedback was used. The results highlight the importance of feedback contingency upon safer decision making in risky contexts, as they showed how a contingent feedback led to faster and safer responses than a control group without any feedback; whereas a non-contingent one gives rise to slower responses and, more importantly, an enhanced risk-taking behavior that could be the cause of undesirable effects on road safety. The feedback effect was even more evident in the appearance of an opposite response bias as a function of contingency and enhanced by learning. These results accord with the theoretical accounts based upon the feedback influence on the threshold level of decision making. Moreover, the effects of feedback may be explained by new proposals focusing on the importance of attentional factors as well as of the mental models people build to react in complex risky scenarios, as a product of feedback and learning. Finally, this research may increase our understanding of the role of feedback in the process of learning safer behavior in complex risky situations.