A tutorial on partially observable Markov decision processes

The partially observable Markov decision process (POMDP) model of environments was first explored in the engineering and operations research communities 40 years ago. More recently, the model has been embraced by researchers in artificial intelligence and machine learning, leading to a flurry of solution algorithms that can identify optimal or near-optimal behavior in many environments represented as POMDPs. The purpose of this article is to introduce the POMDP model to behavioral scientists who may wish to apply the framework to the problem of understanding normative behavior in experimental settings. The article includes concrete examples using a publicly-available POMDP solution package.     

The development and coherence of future-oriented behaviors during the preschool years

Although previous research has identified a number of interesting aspects of future thinking in adults, little is known about the developmental trajectory and coherence of future-oriented behaviors during early childhood. The primary goal of this study was to explore these issues by administering a battery of tasks assessing different aspects of future thinking, including mental time travel, delay of gratification, planning, and prospective memory, to 72 preschoolers. Results revealed that performance on all of the tasks increased significantly between 3 and 5 years of age. Although most tasks were correlated, suggesting “behavioral” coherence, many of these significant correlations dropped out once age and receptive vocabulary were controlled. These results are discussed with respect to theories about, and measurement of, future orientation.     

If you plan,then you can: How reflection helps defensive pessimists pursue their goals

This research examines how the two propensities that underlie defensive pessimism (pessimism and reflection) operate independently to influence the defensive pessimistic process. It investigates the hypothesis that the propensity to reflect, or plan, counteracts the detrimental effects of pessimism by encouraging not only planning, but also the pursuit of those plans. Consistent with these predictions, two studies revealed that the propensity to reflect helped defensive pessimists pursue their plans by (a) increasing goal importance, (b) promoting effort, (c) raising initial expectations, and (d) buffering the anticipated sting of failure. Pessimism hindered performance by increasing anxiety and lowering expectations. Thus, the propensity to reflect counteracts pessimism by not only promoting planning, but also processes that help the pursuit of those plans.

Lifespan changes in working memory in fragile X premutation males

Fragile X syndrome is the world’s most common hereditary cause of developmental delay in males and is now well characterized at the biological, brain and cognitive levels. The disorder is caused by the silencing of a single gene on the X chromosome, the FMR1 gene. The premutation (carrier) status, however, is less well documented but has an emerging literature that highlights a more subtle profile of executive cognitive deficiencies that mirror those reported in fully affected males. Rarely, however, has the issue of age-related declines in cognitive performance in premutation males been addressed. In the present study, we focus specifically on the cognitive domain of working memory and its subcomponents (verbal, spatial and central executive memory) and explore performance across a broad sample of premutation males aged 18–69 years matched on age and IQ to unaffected comparison males. We further tease apart the premutation status into those males with symptoms of the newly identified neurodegenerative disorder, the fragile X-associated tremor/ataxia syndrome (FXTAS) and those males currently symptom-free. Our findings indicate a specific vulnerability in premutation males on tasks that require simultaneous manipulation and storage of new information, so-called executive control of memory. Furthermore, this vulnerability appears to exist regardless of the presence of FXTAS symptoms. Males with FXTAS symptoms demonstrated a more general impairment encompassing phonological working memory in addition to central executive working memory. Among asymptomatic premutation males, we observed the novel finding of a relationship between increased CGG repeat size and impairment to central executive working memory.     

刺激持续时间对注意瞬脱影响的实验分离现象

通过三个实验,研究了注意瞬脱效应以便深入理解注意时间维度上的特性。实验一针对注意瞬脱效应（简称AB效应）和刺激持续显示时间（简称ED）之间的关系,发现SOA或者ED的延长能减小AB效应。实验二的结果再次证明ED的延长能减小AB效应,并且发现ED延长并没有伴随着对目标的加工速度的加快。实验三发现在目标与干扰项没有显著的特征差异时,AB效应随着ED延长而减小的现象消失了,即ED对AB效应的影响出现了实验性的分离。同时还证明突出显示可以通过加快对目标的反应速度来显著的减小AB效应,而且ED因素只有在目标具有突出的视觉特征时才能改变AB效应。     

Motivation through conscious goal setting

The article describes what has been found during 30 years of research by the author and others on the relationship between conscious performance goals and performance on work tasks. This approach is contrasted with previous approaches to motivation theory which stressed physiological, external or subconscious causes of action. The basic contents of goal setting theory are summarized in terms of 14 categories of findings. An applied example is provided.     

Synergistic influences of sensory and central stimuli on non-voluntary rhythmic arm movements

In recent years, neuromodulation of the cervical spinal circuitry has become an area of interest for investigating rhythmogenesis of the human spinal cord and interaction between cervical and lumbosacral circuitries, given the involvement of rhythmic arm muscle activity in many locomotor tasks. We have previously shown that arm muscle vibrostimulation can elicit non-voluntary upper limb oscillations in unloading body conditions. Here we investigated the excitability of the cervical spinal circuitry by applying different peripheral and central stimuli in healthy humans. The rationale for applying combined stimuli is that the efficiency of only one stimulus is generally limited. We found that low-intensity electrical stimulation of the superficial arm median nerve can evoke rhythmic arm movements. Furthermore, the movements were enhanced by additional peripheral stimuli (e.g., arm muscle vibration, head turns or passive rhythmic leg movements). Finally, low-frequency transcranial magnetic stimulation of the motor cortex significantly facilitated rhythmogenesis. The findings are discussed in the general framework of a brain-spinal interface for developing adaptive central pattern generator-modulating therapies.     

A hybrid POMDP-BDI agent architecture with online stochastic planning and plan caching

This article presents an agent architecture for controlling an autonomous agent in stochastic, noisy environments. The architecture combines the partially observable Markov decision process (POMDP) model with the belief-desire-intention (BDI) framework. The Hybrid POMDP-BDI agent architecture takes the best features from the two approaches, that is, the online generation of reward-maximizing courses of action from POMDP theory, and sophisticated multiple goal management from BDI theory. We introduce the advances made since the introduction of the basic architecture, including (i) the ability to pursue and manage multiple goals simultaneously and (ii) a plan library for storing pre-written plans and for storing recently generated plans for future reuse. A version of the architecture is implemented and is evaluated in a simulated environment. The results of the experiments show that the improved hybrid architecture outperforms the standard POMDP architecture and the previous basic hybrid architecture for both processing speed and effectiveness of the agent in reaching its goals.