Reward tampering problems and solutions in reinforcement learning: a causal influence diagram perspective

Can humans get arbitrarily capable reinforcement learning (RL) agents to do their bidding? Or will sufficiently capable RL agents always find ways to bypass their intended objectives by shortcutting their reward signal? This question impacts how far RL can be scaled, and whether alternative paradigms must be developed in order to build safe artificial general intelligence. In this paper, we study when an RL agent has an instrumental goal to tamper with its reward process, and describe design principles that prevent instrumental goals for two different types of reward tampering (reward function tampering and RF-input tampering). Combined, the design principles can prevent reward tampering from being an instrumental goal. The analysis benefits from causal influence diagrams to provide intuitive yet precise formalizations.

     

Texture development during cross rolling of a dual-phase Fe–Cr–Ni alloy: experiments and simulations

Texture development during multi-step cross rolling of a dual-phase Fe–Cr–Ni alloy has been investigated. X-ray diffraction was used to investigate changes in crystallographic texture of both the constituent phases (austenite and ferrite) through changes in orientation distribution function. After deformation, rotated brass (rotated along φ₁, i.e. the sample normal direction ND), along with a weak cube texture was observed in austenite, while a strong rotated cube texture was obtained in ferrite. Texture was also simulated for various strains using a co-deformation model by self-consistent visco-plastic (VPSC) formulation. Simulations showed strong rotated brass texture in austenite and a strongly rotated cube, α-fibre (sample rolling direction RD //<1 1 0>) and γ-fibre (ND //<1 1 1>) in ferrite after highest strain (ε_t = 1.6). VPSC models could not effectively capture the change in crystallographic texture during cross rolling. In ferrite, simulations showed an overestimation of γ-fibre component and an underestimation of rotated cube component. Simulated texture of austenite, on the other hand, showed an overestimation of rotated brass with an absence of cube component. The results are rationalised based on the possible role of shear banding and activation of non-octahedral slip system during cross rolling, both of which are not incorporated in conventional VPSC models.     

Role of Religion and Spirituality in Stress Management Among Nurses

Religion and spirituality play a significant role as coping resources under stressful circumstances. Nursing professionals confront with a variety of stressors repeatedly and are found to employ religious/spiritual coping techniques in managing the negative impact of work stress. The present review explores different religious and spiritual coping strategies utilized by nurses of different socio-cultural and religious backgrounds and highlights the importance of treating religion and spirituality as two separate entities in studying their stress-buffering effect. The MEDLINE, CINAHL, PsycINFO, PsycArticles databases and Google Scholar were searched from 2006 to 2017 with the key words nursing, stress, religious coping, religiosity and spirituality. Various notions of the divine/transcendent aspect of life have led to lack of consensus over a functional definition of religion as well as spirituality. This is found to be the core element of methodological inadequacy in studying individuals’ reliance on religion and spirituality during stress. Further, most of the existing measures do not adequately explore spirituality as a construct independent of religion. Measures should be more culture sensitive to discover culturally enriched religious practices and rituals adopted by individuals belonging to different socio-cultural milieu to overcome stress. The role of religion and spirituality in stress resilience, emotion regulation and burnout among nurses around the globe needs further empirical support. Multiple levels at which religious and spiritual coping may moderate/mediate the relationship between work stress and behavioral/emotional outcomes among human service personnel who face high emotional labor demands should be more comprehensively analyzed.     

Is Synthesis of World Religions Possible?

Introduction

All major religions appear to follow a pattern of social contract with an additional feature of ‘termination of the contract’ and its ‘aftermath’, for the soul. This way, religion is known as ‘dissolution mechanism’ (DM). It contains both social and metaphysical terms, where the former is subsidiary to the latter. The metaphysical term is represented by the mokṣa. A given DM not only explicates the state of soul after the termination of contract, but also it endeavours to point out the nature of the ultimate reality it conceives.

Method

DM provides a geometrical/mathematical picture of the universe to locate the ultimate reality and the individual souls in it; three views of the universe—destruction, creation and sustenance—are obtained for three basic religions or thoughts. Buddhist ‘Nothingness’ forms the psychological background for these views of universe. Architectural designs of worship places, yantras or manḍalas, etc., show both the nature of universe and the location of the ultimate in it. Understanding these figures is the key method to be employed for arriving at synthesis.

Conclusion

Synthesis consists in noting the identical common metaphysical generic essence as running through all the religions, and identifying the unique specific root-essence on such a generic essence. Specific root-essence of a given DM consists in the way the unification of cognitions of the universe is carried out. When the unification of all the specific root-essences on the generic essence is carried out with a further psychological background of Buddhism, we get a specimen of metaphysical synthesis of world religions. Thus, synthesis is a distinct and sure possibility.

     

The influence of second language proficiency on bilingual parallel language activation in Hindi–English bilinguals

We investigated the spontaneous activation of phonologically related words in high and low proficient Hindi–English bilinguals during spoken word processing in an eye-tracking study. Participants listened to spoken words in L1/L2 and looked at a display (consisting of line drawings of phonological cohort of the translation equivalent of the spoken word and unrelated distractors). Both the groups were quick in orienting their attention towards the competitor with the onset of the spoken word. Furthermore, high proficient bilinguals showed higher and earlier activation of the competitor compared to low proficient bilinguals. Cross-language activations were higher in the L2–L1 direction for both the groups. The results strongly suggest language non-selective access of translation in Hindi–English bilinguals in both language directions. We discuss the results with regard to the predictions of the bilingual language processing models and the effect of language proficiency on conceptual access during listening in bilinguals.     

Tomeito or Tomahto: Exploring consumer's accent and their engagement with artificially intelligent interactive voice assistants

Artificially intelligent interactive voice assistants (AIIVAs) are developed to understand language, but there is limited insight into their ability to understand accents. While there have been substantial advancements in understanding multiple languages by AIIVAs, having an understanding of variety of accents is an emerging concern. To address these concerns, we contextualised our study in India, one of the world's most populated and diverse countries with varying accents and dialects. Study 1 collected qualitative data through semi structured interviews with participants, data was subsequently thematically analysed, and a typology was developed with respect to the context of use and consumers' emotional and rational reactions towards AIIVAs when interacting with accents. For Study 2, we implemented the quantitative research method. This was done to reiterate the conceptual model formulated from the qualitative research findings. Findings suggest that positive emotional action has emerged as the most significant factor, followed by rational action and negative emotional action. This study contributes significantly to the theoretical understanding of future consumer behaviour and human-computer interaction trends. It provides practical implications for managers, tech developers, and other companies working and using speech-to-text automatic speech recognition to know that while they train their algorithms with languages, they should be mindful of the diverse accents of their consumers.     

Exploring thematic influences on theme park visitors' satisfaction: An empirical study on Disneyland China

This study aims to explore thematic influences on theme park visitors' satisfaction through user-generated data. To this end, we first used an unsupervised machine learning method, structural topic modeling, and analyzed 112,000 reviews post by visitors to Shanghai Disney Resort from June 16, 2016 to March 4, 2022. Our findings are of great significance for reflecting consumer behavior through user-generated data. Specifically, we find that visitors' satisfaction is highly related to service in the theme park and their playing feeling, and early tourists pay more attention to the experience of specific playing items while later tourists focus on the overall playing experience. In addition, an empirical study is conducted by treating the ratings associated with each review as dependent variable and each topic represented by comments as independent variables, which shows that the relationship between the customer reviews and ratings by tourists becomes less pronounced over time. In other words, as time goes, customers review can reflect their subjective feelings or experience, but the rating is not. We discover the “dynamics” of user-generated data over time and gain a better understanding of the aspects and concerns of visitors' satisfaction over time. The findings of the study contribute to the literature on tourism, service, and consumer behavior while also providing valuable practical implications.     

Screening for major depressive disorders in adolescent psychiatric inpatients with the mood modules from the Primary Care Evaluation of Mental Disorders and the Patient Health Questionnaire.

The mood modules from the Primary Care Evaluation of Mental Disorders (PRIME-MD) and the Patient Health Questionnaire (PHQ) were administered to 17 (52%) female and 16 (48%) male adolescent (13-17 years old) psychiatric inpatients. The internal consistencies of both were good (KR-20 for PRIME-MD=.80, Cronbach coefficient alpha for Patient Health Questionnaire=.85). The correlation between the PRIME-MD and Patient Health Questionnaire total scores was .87 (p<.001), and the point-biserial correlation of both questionnaires' total scores with being diagnosed with a Major Depressive Disorder was .54 (p<.001). Both mood modules appeared to be equally effective in screening for a Major Depressive Disorder.