Reinforcement learning, conditioning, and the brain: Successes and challenges

The field of reinforcement learning has greatly influenced the neuroscientific study of conditioning. This article provides an introduction to reinforcement learning followed by an examination of the successes and challenges using reinforcement learning to understand the neural bases of conditioning. Successes reviewed include (1) the mapping of positive and negative prediction errors to the firing of dopamine neurons and neurons in the lateral habenula, respectively; (2) the mapping of model-based and model-free reinforcement learning to associative and sensorimotor cortico-basal ganglia-thalamo-cortical circuits, respectively; and (3) the mapping of actor and critic to the dorsal and ventral striatum, respectively. Challenges reviewed consist of several behavioral and neural findings that are at odds with standard reinforcement-learning models, including, among others, evidence for hyperbolic discounting and adaptive coding. The article suggests ways of reconciling reinforcement-learning models with many of the challenging findings, and highlights the need for further theoretical developments where necessary. Additional information related to this study may be downloaded from http://cabn.psychonomic-journals.org/content/supplemental.     

Correspondence in children's self-report: Tacting and manding aspects

Four boys and 4 girls (3 to 5 years old) played with as many as three toys chosen from a set of six, and were then asked whether they played with each of the toys. After a baseline in which all children showed high levels of correspondence between reported and actual behavior, reports of play were differentially reinforced, first in an individual and then in a social context. Two children in the individual condition began to report play with all six toys, even though no more than three toys had been played with. When reports of play were reinforced in a group context, 5 children reported play with all six toys. When correspondence was subsequently reinforced, virtually complete correspondence returned and was maintained in a third noncontingent reinforcement condition. Correspondence and lack of correspondence were discussed in terms of self-tacting and distorted tacting or manding.     

Life Experiences Throughout the Lifespan: What Do People Say (or Not) About Them?

Life experiences have been a topic of interest for researchers and clinicians for decades. Current knowledge is rooted in two distinct approaches, i.e., personality psychology and psychosomatics. Whereas the first is interested in ordinary life stories of nonclinical individuals, based on a more qualitative, in-depth, and person-driven approach, psychosomatics stresses negative events, mainly in clinical samples, and presents a more quantitative, general, and construct-driven approach. Consequently, available evidence is dispersed and unrelated and many basic questions remain unanswered. This study aimed to explore occurrence, developmental stage, valence, and impact of life experiences and to analyze critical answering patterns (i.e., “I don’t remember,” missingness). Through a cross-sectional retrospective design, 394 adults from the community answered the Lifetime Experiences Scale, which covers 75 life experiences organized in eight domains (i.e., school, job, health, leisure, living conditions, adverse experiences, achievements, and people and relationships). Occurrence of life experiences varied greatly, and the mean number of experiences reported was approximately 30. Regarding developmental stage, most experiences were reported in just one stage—mainly adulthood—however, some could be considered chronic. Globally, life experiences tended to be clearly rated as positive or as negative; additionally, assessed experiences were mainly appraised as positive. Moreover, participants presented their experiences as significant, rating them as high impact. Overall, critical answering patterns were not very expressive: “I don’t remember” and missing answers were below 2 and 5%, respectively, in the majority of experiences. These findings offer several important new insights, suggesting that life experiences are mainly an idiosyncratic topic.     

Conditioned fear is modulated by D2 receptor pathway connecting the ventral tegmental area and basolateral amygdala

Excitation of the mesocorticolimbic pathway, originating from dopaminergic neurons in the ventral tegmental area (VTA), may be important for the development of exaggerated fear responding. Among the forebrain regions innervated by this pathway, the amygdala is an essential component of the neural circuitry of conditioned fear. The functional role of the dopaminergic pathway connecting the VTA to the basolateral amygdala (BLA) in fear and anxiety has received little attention. In vivo microdialysis was performed to measure dopamine levels in the BLA of Wistar rats that received the dopamine D(2) agonist quinpirole (1 μg/0.2 μl) into the VTA and were subjected to a fear conditioning test using a light as the conditioned stimulus (CS). The effects of intra-BLA injections of the D(1) antagonist SCH 23390 (1 and 2 μg/0.2 μl) and D(2) antagonist sulpiride (1 and 2 μg/0.2 μl) on fear-potentiated startle (FPS) to a light-CS were also assessed. Locomotor performance was evaluated by use of open-field and rotarod tests. Freezing and increased dopamine levels in the BLA in response to the CS were both inhibited by intra-VTA quinpirole. Whereas intra-BLA SCH 23390 did not affect FPS, intra-BLA sulpiride (2 μg) inhibited FPS. Sulpiride's ability to decrease FPS cannot be attributed to nonspecific effects because this drug did not affect motor performance. These findings indicate that the dopamine D(2) receptor pathway connecting the ventral tegmental area and the basolateral amygdala modulates fear and anxiety and may be a novel pharmacological target for the treatment of anxiety.     

Near-term fetuses process temporal features of speech

The perception of speech and music requires processing of variations in spectra and amplitude over different time intervals. Near-term fetuses can discriminate acoustic features, such as frequencies and spectra, but whether they can process complex auditory streams, such as speech sequences and more specifically their temporal variations, fast or relatively slow acoustic variations, is unclear. We recorded the cardiac activity of 82 near-term fetuses (38 weeks GA) in quiet sleep during a silent control condition and four 15 s streams presented at 90 dB SPL Leq: two piano melodies with opposite contours, a natural Icelandic sentence and a chimera of the sentence--all its spectral information was replaced with broadband noise, leaving its specific temporal variations in amplitude intact without any phonological information. All stimuli elicited a heart rate deceleration. The response patterns to the melodies were the same and differed significantly from those observed with the Icelandic sentence and its chimera, which did not differ. The melodies elicited a monophasic heart rate deceleration, indicating a stimulus orienting reflex while the Icelandic and its chimera evoked a sustained lower magnitude response, indicating a sustained attentional response or more focused information processing. A conservative interpretation of the data is that near-term fetuses can perceive sound streams and the rapid temporal variations in amplitude that are specific to speech sounds with no spectral variations at all.     

Maintenance and Transformation of Problematic Self-Narratives: A Semiotic-Dialogical Approach

This study focuses on how the emergence of innovative moments (IMs), which are exceptions to a person’s dominant self-narrative (i.e., his or her usual way of understanding and experiencing), progresses to the construction of a new self-narrative. IMs challenge a person’s current framework of understanding and experiencing, generating uncertainty. When uncertainty is excessively threatening, a semiotic strategy to deal with it often emerges: attenuation of novelty’s meanings and implications by a quick return to the dominant self-narrative. From a dialogical perspective, a dominant voice (which organizes one’s current self-narrative) and a non-dominant or innovative voice (expressed during IMs) establish a cyclical relation, mutual in-feeding, blocking self-development. In this article, we analyze a successful psychotherapeutic case focusing on how the relation between dominant and non-dominant voices evolves from mutual in-feeding to other forms of dialogical relation. We have identified two processes: (1) escalation of the innovative voice(s) thereby inhibiting the dominant voice and (2) dominant and innovative voices negotiating and engaging in joint action.     

Conditioned and unconditioned fear organized in the periaqueductal gray are differentially sensitive to injections of muscimol into amygdaloid nuclei

The lateral and basolateral nuclei of the amygdala (LaA and BLA, respectively) serve as a filter for unconditioned and conditioned aversive information that ascends to higher structures from the brainstem, whereas the central nucleus of the amygdala (CeA) is considered to be the main output for the defense reaction. It has been shown that the dorsal periaqueductal gray (dPAG) is activated by threatening stimuli and has important functional links with the amygdala through two-way anatomical connections. In this work, we examined the influence of chemical inactivation of these nuclei of amygdala on the freezing and escape responses induced by electrical stimulation through electrodes implanted in the dPAG of Wistar rats. Each rat also bore a cannula implanted in the LaA, BLA or CeA for injections of muscimol (0.5 microg/0.5 microL) or its vehicle. The duration of freezing behavior that outlasts electrical stimulation of the dPAG was also measured. On the following day, these animals were submitted to a contextual fear-conditioning using foot shocks as unconditioned stimulus. Conditioned freezing to contextual cues previously associated with foot shocks was also inhibited by injections of muscimol into these amygdaloid nuclei. The contextual conditioned freezing behavior is generated in the neural circuits of conditioned fear in the amygdala. The data obtained also show that injections of muscimol into the three amygdaloid nuclei did not change the aversive threshold of freezing, but disrupted the dPAG post-stimulation freezing. Previous findings that the latter freezing results directly from dPAG stimulation and that it is not sensitive to a context shift suggest that it is unconditioned in nature. Thus, the amygdala can affect some, but not all, aspects of unconditioned freezing. Post-stimulation freezing may reflect the process of transferring aversive information from dPAG to higher brain structures.     

Maternal perinatal representations and their associations with mother–infant interaction and attachment: A longitudinal comparison of Portuguese and Brazilian dyads

Prior research in Western countries (mostly the US, Canada and northern Europe) indicates that mothers' representations are associated with mother–infant interaction quality and their child's attachment security later in the first year. Fewer studies, however, have evaluated whether these associations hold for mother–infant dyads in other countries, such as Brazil and Portugal. Although these countries share a similar language and culture, they differ on societal dimensions that may affect parenting attitudes and mother–infant relationships, such as economic stress, social organisation, social policy, and the availability of services for young families. In this longitudinal study, we followed two independent samples of Brazilian and Portuguese mother–infant dyads from the perinatal period to 12 months post-partum. We assessed mothers' perinatal representations using semi-structured interviews in the first 48 hours after the infant's birth, and evaluated the associations of these representations with mother interaction quality at 9 months and infant attachment at 12 months. Results were similar in each country, corroborating prior research in single Western countries: Mothers with more positive perinatal representations were more sensitive to their infants during free play at 9 months and were more likely to have infants classified as securely attached at 12 months.     

Building reciprocity: From safety‐net to social transformation programmes

Topics of societal concern such as mental health and poverty reduction increasingly require action programmes that operate within broad psychosocial and social justice perspectives. Models of practice centred in individual needs, although important, are not powerful enough to bring about social change when they operate in isolation. In this article, we present the findings resulting from the observation of programmes engaged in collaborating with socio‐economically disadvantaged individuals, families, and communities. The programmes selected for study were nationally or internationally recognized for the quality and innovation of their methodologies or for having been subjected to scientific attention; some met both criteria. Altogether, 15 programmes were visited, in North and South America and Europe. Through a grounded theory methodology, the processes of data collection and analysis led to the development of a theoretical framework that identifies a continuum of programmes aimed at supporting the development of individuals, families, and communities and that has at its core the central process of building reciprocity. This article presents and describes the continuum of programmes and how each type relates to the process of building reciprocity and establishes links with other relevant and significant concepts in the framework. Finally, implications for further research are explored.     

Innovative Moments as Developmental Change Levels: A Case Study on Meaning Integration in the Treatment of Depression

Innovative moments are alternative meanings to the problems expressed by clients and are organized into three levels. This study analyzes if these levels are not only change markers related to symptomatology outcomes, but also gradual steps in meaning development. A thematic analysis was carried out in a case of depression treated with narrative therapy. Results revealed three themes that evolved from level 1 to level 3. Levels 1 and 2 corresponded to one specific theme and almost all level 3 integrated different themes (polythematic). Level 3 innovative moments might be an integrative bridge allowing the construction of a more adaptive meaning framework.