Object affordance based multimodal fusion for natural Human-Robot interaction

Spoken language based natural Human-Robot Interaction (HRI) requires robots to have the ability to understand spoken language, and extract the intention-related information from the working scenario. For grounding the intention-related object in the working environment, object affordance recognition could be a feasible way. To this end, we propose a dataset and a deep CNN based architecture to learn the human-centered object affordance. Furthermore, we present an affordance based multimodal fusion framework to realize intended object grasping according to the spoken instructions of human users. The proposed framework contains an intention semantics extraction module which is employed to extract the intention from spoken language, a deep Convolutional Neural Network (CNN) based object affordance recognition module which is applied to recognize human-centered object affordance, and a multimodal fusion module which is adopted to bridge the extracted intentions and the recognized object affordances. We also complete multiple intended object grasping experiments on a PR2 platform to validate the feasibility and practicability of the presented HRI framework.     

Affordances and the Nature of Perceptual Content

Abstract

According to John McDowell, representational perceptual content is conceptual through and through. This paper criticizes this view by claiming that there is a certain kind of representational and non‐conceptual perceptual content that is sensitive to bodily skills. After a brief introduction to McDowell’s position, Merleau‐Ponty’s notion of body schema and Gibson’s notion of affordance are presented. It is argued that affordances are constitutive of representational perceptual content, and that at least some affordances, the so‐called ‘conditional affordances’, are essentially related to the body schema. This means that the perceptual content depends upon the nature of the body schema. Since the body schema does not pertain to the domain that our conceptual faculties operate upon, it is argued that this kind of perceptual content cannot be conceptual. At least some of that content is representational, yet it cannot feature as non‐demonstrative conceptual content. It is argued that if it features as demonstrative conceptual content, it has to be captured by private concepts. Since McDowell’s theory does not allow for the existence of a private language, it is concluded that at least some representational perceptual content is non‐conceptual.     

The Critical Point to Step into a Hole is Similar in Young and Older Adults

Abstract

The avoidance of a hole in the pathway while walking has been systematically investigated; however, depending on the dimensions of the hole, the option to avoid it is infeasible, and it is necessary to use the so-called accommodation strategy to step into the hole. We investigated the critical point between the avoidance and accommodation strategies when dealing with a hole in the ground during locomotion of young and older adults. Young and older adults performed two tasks: verbal estimation and walking. We used holes of different lengths and constant depth (12?cm). In the verbal estimation task, participants stood and looked at each hole and verbally respond if they would step into or avoid it. In the walking task, they walked and chose to either step or avoid the hole. Both age groups preferred to step into the hole when it was larger than 1.3 times their foot length in both tasks. The perception of affordances of young and older adults to step into a hole was similar, and it was unaffected by the investigated tasks. Thus, our participants preferred to have a safety margin that was large enough to guarantee that the whole foot would accommodate within the hole.     

基于客体的一致性效应的产生机制

采用具有不同深度信息的剪影和图片刺激为材料,要求被试完成形状判断任务,探究基于客体的一致性效应的产生机制。实验1中客体刺激呈现于屏幕中央位置;实验2进一步增强了刺激空间呈现的左右位置倾向;实验3则通过交叉手的范式分离了反应位置和反应手不同的编码对一致性效应的作用。结果发现:当刺激不存在显著的左右位置信息时,剪影刺激出现了一致性效应,图片则没有;而当刺激的空间位置信息显著时,剪影和图片均出现了一致性效应,这一效应在反应位置和反应手编码分离后仍然存在。因此得出结论:空间位置编码假说对解释基于客体的一致性效应的产生有重要作用。     

示能性:基于镜像神经元视角的理解

示能性(affordance)概念解释了人的行为与物体功能之间互动、互补的关系。但是,有关示能性生理基础却很少被提及。镜像神经元的发现为示能性生理基础提供了可能的解释。文章根据研究将示能性分为结构示能性和功能示能性,探讨了不同示能性和镜像神经元之间的关系,并提出不同神经通路中的镜像神经元是示能性生理基础的结论。背-背侧分流中的镜像神经元是结构示能性的神经基础,而腹-背侧分流中的镜像神经元是功能示能性的神经基础。     

Conceptual and physical object qualities contribute differently to motor affordances

Priming studies have demonstrated that an object’s intrinsic and extrinsic qualities (size, orientation) influence subsequent motor behavior thus suggesting that these object qualities ‘afford’ actions that are congruent with the prime. We present four experiments that aim to evaluate the relative effect of conceptual and physical object qualities on action priming. In Experiment 1 equally graspable known and unknown tools are presented as primes. In Experiment 2 the primes depict high versus low graspable unfamiliar tools, and in Experiments 3 and 4 we present simple graspable shapes versus high graspable unfamiliar or familiar tools respectively. In all experiments the (unrelated) task consists of a timed motor response to the direction of a centrally placed arrow that is superimposed on the prime. Whereas tool familiarity reveals no significant difference on reaction time (Exp 1), responses to high graspable unfamiliar tools (Exp 2) and simple graspable shapes (Exps 3 and 4) are significantly faster. We conclude that motor affordances are most readily determined by object qualities that depend on the object’s physical appearance provided by visual information. Conceptual information about the stimuli, such as semantic category or stored knowledge about its function and associated movements, does not appear to produce detectable effects of action priming in this paradigm.     

What Can be Done with an Egg? Creativity,Material Objects,and the Theory of Affordances

This article offers a reflection on the role of material objects in the creative process and explores the potential links between creativity and the theory of affordances (Gibson, 1986 ), conceptualized from a sociocultural perspective. From this standpoint, creativity can be defined as a process of perceiving, exploiting, and “generating” novel affordances during socially and materially situated activities. Illustrations are offered for each of the above from a research project investigating traditional Easter egg decoration activities in rural Romania. This brief case study exemplifies the discovery of existing action potentials, the generation of objects with novel affordances, and the transgression of conventional procedures, all resulting in creative forms of expression. In the end, some conclusions are drawn regarding the ways in which an affordance theory of creativity can enrich our understanding of the phenomenon and contribute to the development of a new program of research concerned with situated and distributed creative acts.     

The Influence of Visually Dangerous Information on Imitation in Children

Abstract

Previous studies have demonstrated that when perceiving the actions of another agent, individuals will automatically imitate those observed actions. This study investigated how children’s imitation of physical actions was influenced by either visually neutral or visually dangerous information. Participants were presented with a series of pictures in which an agent was reaching towards either a neutral object or a dangerous object. Results showed that the imitation effect occurred when the agent was observed reaching and grasping a neutral object. However, this effect was not present when the agent’s hand was observed reaching towards or grasping, the non-handle side of a dangerous object. These results suggest children can predict potential behavioral consequences and adjust their imitative action depending on the perceived danger of the action.