Bayesian parameter estimation in the Expectancy Valence model of the Iowa gambling task

The purpose of the popular Iowa gambling task is to study decision making deficits in clinical populations by mimicking real-life decision making in an experimental context. Busemeyer and Stout [Busemeyer, J. R., & Stout, J. C. (2002). A contribution of cognitive decision models to clinical assessment: Decomposing performance on the Bechara gambling task. Psychological Assessment, 14, 253-262] proposed an “Expectancy Valence” reinforcement learning model that estimates three latent components which are assumed to jointly determine choice behavior in the Iowa gambling task: weighing of wins versus losses, memory for past payoffs, and response consistency. In this article we explore the statistical properties of the Expectancy Valence model. We first demonstrate the difficulty of applying the model on the level of a single participant, we then propose and implement a Bayesian hierarchical estimation procedure to coherently combine information from different participants, and we finally apply the Bayesian estimation procedure to data from an experiment designed to provide a test of specific influence.     

The exponential learning equation as a function of successful trials results in sigmoid performance

While the exponential learning equation, indicating a gradually diminishing improvement, is one of the standard equations to describe learning, a sigmoid behavior with initially increasing then decreasing improvement has also been suggested. Here we show that the sigmoid behavior is mathematically derived from the standard exponential equation when the independent variable of the equation is restricted to the successful trials alone. It is suggested that for tasks promoting success-based learning, performance is better described by the derived sigmoid curve.     

RIGHT ACT, VIRTUOUS MOTIVE

Abstract: The concepts of virtue and right action are closely connected, in that we expect people with virtuous motives to at least often act rightly. Two well-known views explain this connection by defining one of the concepts in terms of the other. Instrumentalists about virtue identify virtuous motives as those that lead to right acts; virtue-ethicists identify right acts as those that are or would be done from virtuous motives. This essay outlines a rival explanation, based on the "higher-level" account of virtue defended in the author's Virtue, Vice, and Value . On this account rightness and virtue go together because each is defined by a (different) relation to some other, more basic moral concept. Their frequent coincidence is therefore like a correlation between A and B based not on either's causing the other but on their being joint effects of a single common cause.     

A PARADIGM FOR OPERANT CONDITIONING IN BLOW FLIES (PHORMIA TERRAE NOVAE ROBINEAU‐DESVOIDY, 1830)

An operant conditioning situation for the blow fly (Protophormia terrae novae) is described. Individual flies are trained to enter and reenter a hole as the operant response. Only a few sessions of contingent reinforcement are required to increase response rates. When the response is no longer followed by food, the rate of entering the hole decreases. Control procedures revealed that rate of responding is not a simple overall result of feeding or of aging. The flies entered into the hole only if the response was required to obtain the food.     

Distribution-free tests of stochastic dominance for small samples

One variable is said to “stochastically dominate” another if the probability of observations smaller than x is greater for one variable than the other, for all x. Inferring stochastic dominance from data samples is important for many applications of econometrics and experimental psychology, but little is known about the performance of existing inferential methods. Through simulation, we show that three of the most widely used inferential methods are inadequate for use in small samples of the size commonly encountered in many applications (up to 400 observations from each distribution). We develop two new inferential methods that perform very well in a limited, but practically important, case where the two variables are guaranteed not to be equal in distribution. We also show that extensions of these new methods, and an improved version of an existing method, perform quite well in the original, unlimited case.     

A Bayesian-optimal principle for learner-friendly adaptation in learning games

Adaptive learning games should provide opportunities for the student to learn as well as motivate playing until goals have been reached. In this paper, we give a mathematically rigorous treatment of the problem in the framework of Bayesian decision theory. To quantify the opportunities for learning, we assume that the learning tasks that yield the most information about the current skills of the student, while being desirable for measurement in their own right, would also be among those that are efficient for learning. Indeed, optimization of the expected information gain appears to naturally avoid tasks that are exceedingly demanding or exceedingly easy as their results are predictable and thus uninformative. Still, tasks that are efficient for learning may be experienced as too challenging, and the resulting failures can lower motivation. Therefore, in addition to quantifying the expected informational benefit for learning of any prospective task to be presented next, we also model the expected motivational cost of its presentation, measured simply as the estimated probability of failure in our example model. We propose a “learner-friendly” adaptation algorithm that chooses the learning tasks by optimizing the expected benefit divided by the expected cost. We apply this algorithm to a Rasch-like student model implemented within a real-world application and present initial results of a pilot experiment.     

Implicit learning of social predictions

Humans exchange a range of nonverbal social signals in every interaction. It is an open question whether people use these signals, consciously or unconsciously, to guide social behavior. This experiment directly tested whether participants could learn to predict another person’s behavior using nonverbal cues in a single interaction, and whether explicit knowledge of the cue-outcome relationship was necessary for successful prediction. Participants played a computerized game of rock-paper-scissors against an avatar they believed was another participant. Sometimes the avatar generated a predictive facial cue before the play. On these trials, participants’ win-frequency increased over time, even if they did not acquire explicit knowledge of the predictive cue. The degree to which participants could predict the avatar (wins on cued trials) related to their self-reported liking of the avatar. These findings demonstrate the importance of implicit associative learning mechanisms in guiding social behavior on a moment-to-moment basis during interaction.     

Learned preference for a hedonically negative flavor is observed after pairings with positive post-ingestion consequences rather than with a palatable flavor

In two experiments, thirsty rats consumed a compound of sucrose and a non-preferred flavor. In Experiment 1, a conditioned preference was observed in the experimental group when animals were tested both thirsty and hungry, but not when they were tested just thirsty. Animals in the control group, which experienced the flavor and the sucrose unpaired, never showed a preference. Experiment 2 replicated the absence of a preference in the experimental group when rats were tested thirsty, but provided evidence that a flavor-taste association had been formed during training. After conditioning, sucrose was paired with LiCl in group Dev whereas it was unpaired in group NonDev. The sucrose devaluation produced a decrease in CS preference in group Dev, and an increment in group NonDev. Taken together, these results show that preference for a non-preferred flavor can be readily observed after pairings with the positive consequences of the US (calories or absence of an expected illness) rather than with a palatable flavor.     

Applying memories of reinforcement outcomes mainly to Pavlovian conditioning

A theory devised initially on the basis of instrumental reward schedule data, such as the PREE, was extended to deal with various Pavlovian findings. These Pavlovian findings include blocking, unblocking, relative validity, positive and negative patterning, renewal, reinstatement, reacquisition, and inhibition. In addition, the sequential model was discussed in relation to the rule-learning view of Hulse and Dorsky (1977) employing data from a report by Bloom and Capaldi (1961).     

Song recognition in zebra finches: Are there sensitive periods for song memorization?

Male zebra finches learn to sing songs that they hear between 25 and 65 days of age, the sensitive period for song learning. In this experiment, male and female zebra finches were exposed to zebra finch songs either before (n = 9) or during (n = 4) the sensitive period. Following song exposure, recognition memory for the songs was assessed with an operant discrimination between familiar and novel songs. Zebra finches that were exposed to songs between 22 and 30 days of age discriminated between familiar and novel songs; zebra finches exposed to songs from 9 to 17 days of age did not. Failure to memorize songs heard prior to the sensitive period may contribute to the exclusion of those songs from the repertoire of songbirds.