Interactions between deliberation and delay-discounting in rats

When faced with decisions, rats sometimes pause and look back and forth between possible alternatives, a phenomenon termed vicarious trial and error (VTE). When it was first observed in the 1930s, VTE was theorized to be a mechanism for exploration. Later theories suggested that VTE aided the resolution of sensory or neuroeconomic conflict. In contrast, recent neurophysiological data suggest that VTE reflects a dynamic search and evaluation process. These theories make unique predictions about the timing of VTE on behavioral tasks. We tested these theories of VTE on a T-maze with return rails, where rats were given a choice between a smaller reward available after one delay or a larger reward available after an adjustable delay. Rats showed three clear phases of behavior on this task: investigation, characterized by discovery of task parameters; titration, characterized by iterative adjustment of the delay to a preferred interval; and exploitation, characterized by alternation to hold the delay at the preferred interval. We found that VTE events occurred during adjustment laps more often than during alternation laps. Results were incompatible with theories of VTE as an exploratory behavior, as reflecting sensory conflict, or as a simple neuroeconomic valuation process. Instead, our results were most consistent with VTE as reflecting a search process during deliberative decision making. This pattern of VTE that we observed is reminiscent of current navigational theories proposing a transition from a deliberative to a habitual decision-making mechanism.     

Artificial organisms as tools for the development of psychological theory: Tolman’s lesson

In the 1930s and 1940s, Edward Tolman developed a psychological theory of spatial orientation in rats and humans. He expressed his theory as an automaton (the "schematic sowbug") or what today we would call an "artificial organism." With the technology of the day, he could not implement his model. Nonetheless, he used it to develop empirical predictions which tested with animals in the laboratory. This way of proceeding was in line with scientific practice dating back to Galileo. The way psychologists use artificial organisms in their work today breaks with this tradition. Modern "artificial organisms" are constructed a posteriori, working from experimental or ethological observations. As a result, researchers can use them to confirm a theoretical model or to simulate its operation. But they make no contribution to the actual building of models. In this paper, we try to return to Tolman's original strategy: implementing his theory of "vicarious trial and error" in a simulated robot, forecasting the robot's behavior and conducting experiments that verify or falsify these predictions.     

Vicarious dissonance: attitude change from the inconsistency of others

Three studies support the vicarious dissonance hypothesis that individuals change their attitudes when witnessing members of important groups engage in inconsistent behavior. Study 1, in which participants observed an actor in an induced-compliance paradigm, documented that students who identified with their college supported an issue more after hearing an ingroup member make a counterattitudinal speech in favor of that issue. In Study 2, vicarious dissonance occurred even when participants did not hear a speech, and attitude change was highest when the speaker was known to disagree with the issue. Study 3 showed that speaker choice and aversive consequences moderated vicarious dissonance, and demonstrated that vicarious discomfort--the discomfort observers imagine feeling if in an actor's place--was attenuated after participants expressed their revised attitudes.     

An automated runway: Responding without postreinforcement goal tracking

The behaviors of rodents in manual runways are subject to experimenter variability and error and to the stimulus properties of handling. Automated shuttle runways are limited by the shuttle response itself. The response is difficult to learn and maintain because it requires that subjects leave an area associated with reinforcement. This report describes an automated analogue of the traditional straight alley that features unidirectional responding and a goal area physically removed from the runway after each trial. A series of studies demonstrate that rats and gerbils quickly acquire and maintain a stable appetitive running response. This response also provides a sensitive measure for the effects of conditioned fear.     

Transfer of metacognitive skills and hint seeking in monkeys

Metacognition is knowledge that can be expressed as confidence judgments about what one knows (monitoring) and by strategies for learning what one does not know (control). Although there is a substantial literature on cognitive processes in animals, little is known about their metacognitive abilities. Here we show that rhesus macaques, trained previously to make retrospective confidence judgments about their performance on perceptual tasks, transferred that ability immediately to a new perceptual task and to a working memory task. We also show that monkeys can learn to request "hints" when they are given problems that they would otherwise have to solve by trial and error. This study demonstrates, for the first time, that nonhuman primates share with humans the ability to monitor and transfer their metacognitive ability both within and between different cognitive tasks, and to seek new knowledge on a need-to-know basis.     

Learning from delayed feedback: neural responses in temporal credit assignment

When feedback follows a sequence of decisions, relationships between actions and outcomes can be difficult to learn. We used event-related potentials (ERPs) to understand how people overcome this temporal credit assignment problem. Participants performed a sequential decision task that required two decisions on each trial. The first decision led to an intermediate state that was predictive of the trial outcome, and the second decision was followed by positive or negative trial feedback. The feedback-related negativity (fERN), a component thought to reflect reward prediction error, followed negative feedback and negative intermediate states. This suggests that participants evaluated intermediate states in terms of expected future reward, and that these evaluations supported learning of earlier actions within sequences. We examine the predictions of several temporal-difference models to determine whether the behavioral and ERP results reflected a reinforcement-learning process.     

Effects of vicarious reward and punishment on social imitation in chronic psychotics

Despite considerable evidence that modeling procedures provide effective methods for the acquisition and change of behavior, there is very little research on the effects of vicarious reward, and none on the effects of vicarious punishment, on social imitation in chronic psychotics. Two studies (Goldstein et al., 1973: Gutride, Goldstein and Hunter, 1973) demonstrated the superiority of vicarious reward over no-treatment with adult psychotics. However, in neither study was a model no-consequences group included in order to control for the effects of observation per se; therefore, such findings cannot be clearly attributed to vicarious reinforcement (Thelen and Rennie, 1972). The only other study known to bear on this issue revealed no differences in initial learning between a model-rewarded and a model no-consequences group (Olson, 1971).The present study included model no-consequences control groups in an attempt to examine the effects of vicarious reward and punishment on subsequent interview behavior in chronic psychotics. Drawing on the larger body of vicarious learning research (Bandura, 1969). it was hypothesized that such patients demonstrate higher levels of socially appropriate behavior after observation of a model who is (a) rewarded for appropriate behavior or (b) punished for inappropriate responses, and lower levels of appropriate behavior after exposure to a model who is (c) rewarded for inappropriate behavior or (d) punished for appropriate responses, relative to conditions in which the same modeled behaviors elicit no contingent consequences.     

EFFECTS OF CHOOSING ACADEMIC ASSIGNMENTS ON A STUDENT WITH ATTENTION DEFICIT HYPERACTIVITY DISORDER

The effects of choosing academic assignments on the undesirable behaviors manifested by a second-grade student with attention deficit hyperactivity disorder were analyzed. This study extended Dunlap et al.'s (1994) research on choice making as a form of antecedent control. A reversal design showed that undesirable behaviors decreased when the student was given a choice of academic assignments.     

With age comes wisdom: decision making in younger and older adults

In two experiments, younger and older adults performed decision-making tasks in which reward values available were either independent of or dependent on the previous sequence of choices made. The choice-independent task involved learning and exploiting the options that gave the highest rewards on each trial. In this task, the stability of the expected reward for each option was not influenced by the previous choices participants made. The choice-dependent task involved learning how each choice influenced future rewards for two options and making the best decisions based on that knowledge. Younger adults performed better when rewards were independent of choice, whereas older adults performed better when rewards were dependent on choice. These findings suggest a fundamental difference in the way in which younger adults and older adults approach decision-making situations. We discuss the results in the context of prominent decision-making theories and offer possible explanations based on neurobiological and behavioral changes associated with aging.     

Errorless learning of a conditional temporal discrimination

In the present study we extended errorless learning to a conditional temporal discrimination. Pigeons' responses to a left-red key after a 2-s sample and to a right-green key after a 10-s sample were reinforced. There were two groups: One learned the discrimination through trial and error and the other through an errorless learning procedure. Then, both groups were presented with three types of tests. First, they were exposed to intermediate durations between 2 s and 10 s, and given a choice between both keys (stimulus generalization test). Second, a delay from 1 s to 16 s was included between the offset of the sample and the onset of the choice keys (delay test). Finally, pigeons learned a new discrimination in which the stimuli were switched (reversal test). Results showed that pigeons from the Errorless group made significantly fewer errors than those in the Trial-and-Error group. Both groups performed similarly during the stimulus generalization test and the reversal test, but results of the delay test suggested that, on long stimulus trials, responding in the errorless training group was less disrupted by delays.     

Females in the forefront: time-based intervention effects on impulsive choice and interval timing in female rats

Impulsive choice has been implicated in substance abuse, gambling, obesity, and other maladaptive behaviors. Deficits in interval timing may increase impulsive choices, and therefore, could serve as an avenue through which suboptimal impulsive choices can be moderated. Temporal interventions have successfully attenuated impulsive choices in male rats, but the efficacy of a temporal intervention has yet to be assessed in female rats. As such, this experiment examined timing and choice behavior in female rats, and evaluated the ability of a temporal intervention to mitigate impulsive choice behavior. The temporal intervention administered in this study was successful in reducing impulsive choices compared to a control group. Results of a temporal bisection task indicated that the temporal intervention increased long responses at the shorter durations. Further, results from the peak trials within the choice task combined with the progressive interval task suggest that the intervention increased sensitivity to delay and enhanced timing confidence. Overall, these results indicate that a temporal intervention can be a successful avenue for reducing impulsive choice behavior in female rats, and could contribute to the development of behavioral interventions to prevent impulsive choice and maladaptive behaviors that can be applied to both sexes.     

Within-session reversal learning in rhesus macaques (<Emphasis Type="Italic">Macaca mulatta</Emphasis>)

In a midsession reversal (MSR) task, animals are typically presented with a simple, simultaneous discrimination (S1+, S2?) where contingencies are reversed (S1?, S2+) half-way through each session. This paradigm creates multiple, relevant cues that can aid in maximizing overall reinforcement. Recent research has shown that pigeons show systematic anticipatory and perseverative errors across the session, which increase as a function of proximity to the reversal trial. This behavior has been theorized to indicate primary control by temporal cues across the session, instead of the cues provided by recent reinforcement history that appear to control behavior shown by humans. Rats, however, appear to be guided by recent reinforcement history when tested in an operant context, thereby demonstrating behavior that parallels that seen in humans, but they appear to be guided by temporal cues when tested in an open-field apparatus, showing behavior more akin to that seen in pigeons. We tested rhesus macaques (Macaca mulatta) on the MSR with a computerized simultaneous visual discrimination to assess whether they would show errors indicative of control by time or by recent reinforcement history. When a single reversal point occurred midsession, rhesus macaques showed no anticipation of the reversal and a similar level of perseveration to rats tested in an operant setting. Nearly identical results also were observed when the monkeys were trained with a single, variable reversal point or with multiple, variable reversal points within a session. These results indicate that temporal cues are not guiding response flexibility in rhesus macaque visual discrimination.     

Animal welfare and individual characteristics: a conversation against speciesism

It seems impossible for a human being not to have some point of view concerning nonhuman animal (hereafter animal) welfare. Many people make decisions about how humans are permitted to treat animals using speciesist criteria, basing their decisions on an individual's species membership rather than on that animal's individual characteristics. Although speciesism provides a convenient way for making difficult decisions about who should be used in different types of research, we argue that such decisions should rely on an analysis of individual characteristics and should not be based merely on species membership. We do not argue that the concept of species is never useful or important. To make our points, we present a conversation among a skeptic, an agnostic, and a proponent of the view that our moral obligations to an animal must be based on an analysis of that individual's characteristics. In the course of the discussion, concepts such as personhood, consciousness, cognitive ability, harm, and pain are presented, because one's understanding of these concepts informs his or her ethical decisions about the use of animals by humans.     

The evolution of mandler's conceptual primitives (image-schemas) as neural mechanisms for space-time simulation structures

The purpose of this article is to describe parallels between Mandler's theory of conceptual development in infancy, and my theory of the maximum-power principle evolution of the neuro-algorithmic organization of the brain and cultural-level mental models. A maximum power principle prey-predator scenario depicts the dynamic selective origins of neuro-algorithms underlying Mandler's various image-schemas (conceptual primitives). It is proposed that (a) image-schemas are inherited space-time simulation structures that originate in cerebellar state estimating functions which spread to the mapping systems of the cerebrum; (b) image-schemas do not come about as the result of perceptual analytic abstractive processes as Mandler has proposed, but undergo refinement in ontogeny through experience-expectant development involving perceptual analysis; (c) Mandler's image-schemas are “state-primitives” fed forward from the cerebellum to provide tracking routines for perceptual analysis, and rapid simulations constituting representation, prediction, and control; and (d) vicarious trial and error (VTE) can be interpreted as pragmatic vector simulation using image-schematic simulation structures. A scenario connecting image-schemas with Pleistocene era auditory-vocal system evolution which supports Mandler's developmental connection between image-schemas and language is proposed. It is concluded that Mandler's theory of conceptual development is essentially correct. However, it is suggested that the further articulation of Mandler's theory would benefit from parallel statement of the evolution of underlying brain mechanisms.     

物质成瘾及其戒除:基于反转学习的视角

物质成瘾与反转学习损伤密切相关,成瘾者往往不能灵活地适应变化的刺激—结果的联结,这可能进一步加剧成瘾者的物质使用。近年来研究发现,物质成瘾者的反转学习相关的腹外侧前额和背外侧前额等脑区激活异常,这些异常与成瘾者的冲动性和强迫性有关。此外,个体的反转学习能力对其成瘾行为具有一定预测性。今后应增加对不同类型物质成瘾者的反转学习脑机制及物质相关线索对成瘾者反转学习影响的研究,并且进一步探讨成瘾者的冲动性和强迫性对其反转学习的调节及个体反转学习能力对其成瘾行为的预测。     

The differential involvement of the prelimbic and infralimbic cortices in response conflict affects behavioral flexibility in rats trained in a new automated strategy-switching task

To assess the role of the prelimbic (PL) and infralimbic (IL) cortices in mediating strategy switching, rats were trained in a new automated task in a Y-maze allowing a careful analysis of rats' behavior. In this situation, rats can only use two egocentric (Right, Left) and two visual (Light, Dark) strategies. In the first experiment, rats with PL, IL, or PL/IL lesions were compared with sham-operated rats when trained to reach a criterion of 10 consecutive correct responses with a light strategy before being trained with a response strategy (rule shifting), and finally with the reversed response strategy (reversal). In the second experiment, sham-operated and PL-lesioned rats had their first two strategy switches in the reverse order, which was followed by a second rule shifting and reversal. The results indicate that lesions did not affect initial acquisition, but impaired the first rule shifting and reversal. Thorough analyses of rats' performance indicate that lesioned rats were still able to demonstrate some behavioral flexibility but have difficulties in solving response conflicts, which in turn may affect behavioral flexibility. Both areas were differentially involved in the resolution of response conflict, with the IL involved in the choice of strategy previously known to be nonvalid, and the PL in the selection and maintenance of that strategy.     

The Life Career Game

Vocational counseling of adolescents is difficult because it requires their evaluation of situations that are unfamiliar and unimportant to many of them. Games with simulated environments are presented as one means of overcoming the discrepancy between the adolescent and the adult world. An example of such a game, in which students plan the life of a fictitious student and receive “feedback” on the possible consequences of their decisions is described. Field testing with the game indicates that it can arouse student interest, communicate factual information about career decision-making, and give young people a realistic, if vicarious, experience of certain aspects of adult life.     

Acceptance and recall of written prosocial,neutral, and aggressive modeling cues1

Nine groups of 50 college students responded to acceptance (imitation) and recall questions after being exposed to one of three written modeling sequences (prosocial, neutral, or aggressive), each of which included one of three types of vicarious consequences (positive, neutral, or negative). The results provided strong support for the theoretical predictions regarding the influence of written vicarious consequences on acceptance and recall of modeled behaviors. Regardless of the type of story, positive vicarious consequences led to a significantly higher level of acceptance of the modeled actions than did negative vicarious consequences. The effects of neutral vicarious consequences on acceptance varied with the nature of the modeling story. Although positive and negative vicarious consequences generally led to higher levels of overall recall than did neutral vicarious consequences, patterns of overall recall varied across the three types of stories. Similarly, analyses of the three components (details, behaviors, consequences) of the overall recall measure indicated that recall patterns were inconsistent across the three types of stories. Thus, the results of the present study indicate the importance of considering both vicarious consequences and the situations in which these consequences occur. Furthermore, the effects of written vicarious consequences appear to parallel those of live and/or taped consequences. Continued investigation of these effects is highly warranted.     

The overtraining reversal effect in rats: A function of task difficulty

Two experiments were conducted with rats. The first study showed that in the T-maze a brightness discrimination learning problem was more difficult than a spatial learning problem, but in a Ross-maze a brightness problem was less difficult than a spatial task. T-maze brightness and Ross-maze spatial tasks were found to be of equal difficulty.

In the second experiment rats were trained either on a brightness or spatial discrimination reversal problem in the Ross-maze. It was found that overtraining facilitated reversal performance in the spatial task but not in the brightness problem. The theoretical implications of these results were discussed.     

Studies in observational paired-associate learning: Informational,social, and individual difference variables

Four experiments examined the relative importance of informational (proportion of correct responses and kinds of errors emitted by a model), social (model competency, sex of model, video vs. audio taped model), and individual difference (sex of subject, grade) variables in observational paired-associate learning. In Experiments I–III, vicarious subjects received cycles of study-model-test trials, while direct subjects were given the same sequences with intervening test or stimulus familiarization trials. In Experiment IV, vicarious subjects received cycles of study-test-model-test trials, while direct subjects received the cycles with a test trial replacing the model trial. No confirmation was provided on test and model trials. Whereas the effects attributable to social and individual difference variables were generally negligible, mere accuracy of the model's responses repeatedly covaried with performance on the last test trial of each cycle. Conditional analyses established that (1) vicarious facilitation is comparable across cycles and localized in items responded to incorrectly on immediately preceding test trials, and (2) observers learn fewer incorrect than correct model responses. Vicarious groups performed at reliably higher levels than direct subjects on model correct but not incorrect items. The results strongly suggest a close correspondence between direct and vicarious verbal learning principles and mechanisms.