Cross Talk Between Native and Second Languages: Partial Activation of an Irrelevant Lexicon

Bilingualism provides a unique opportunity for exploring hypotheses about how the human brain encodes language. For example, the "input switch" theory states that bilinguals can deactivate one language module while using the other. A new measure of spoken language comprehension, headband-mounted eyetracking, allows a firm test of this theory. When given spoken instructions to pick up an object, in a monolingual session, late bilinguals looked briefly at a distractor object whose name in the irrelevant language was initially phonetically similar to the spoken word more often than they looked at a control distractor object. This result indicates some overlap between the two languages in bilinguals, and provides support for parallel, interactive accounts of spoken word recognition in general.     

When left feels right: asymmetry in the affordance effect

Previous research has demonstrated the existence of the so-called affordance effect (faster response when the visual affordance of a graspable object (e.g., a pan) corresponds to the response location). It has been argued that the effect is due to abstract spatial coding of the position of the handle relative to the object instead of the grasping affordance. Our experiment tested the hypothesis that the affordance effect is not caused solely by abstract spatial coding but also by specific motor activation in response to the visual affordance. We assumed that, in the case of abstract spatial codes, response location and not hand distinction would be the critical factor for producing an affordance effect. In our experiment, bi-manual crossed/uncrossed responses to left/right symbols superimposed on a picture of a pan were used. The task was to attend to the symbol and to press a corresponding left/right button. Affordances of the pan were manipulated. A three-way interaction between side of affordance (left/right), mode of response (crossed/uncrossed hands), and response-affordance correspondence (corresponding/non-corresponding) showed a correspondence effect in the right affordance condition with hands uncrossed and no correspondence effect with hands crossed. The correspondence effect was obtained in the left affordance condition across hand positions that differed only by their magnitude. Overall, the results suggest that both mechanisms (grasping affordance and spatial codes) differentially contribute to the processing of an object with a graspable handle, depending on the affordance side and response hand.     

Memory for intention-related material presented in a to-be-ignored channel

Three experiments were conducted to investigate the fate of intention-related material processed in a to-be-ignored channel. Participants were given an intention to respond to cues in a visual-processing stream while simultaneously trying to ignore information being presented in an auditory stream. Subsequent to the ongoing activity, a surprise recognition test for information presented in the to-be-ignored auditory modality was administered. As compared with comparable neutral information, corrected recognition memory for intention-related material was significantly better, depending on the type of event-based prospective memory task. These results suggest that holding certain kinds of intentions can bias attentional processes in a manner consistent with a perceptual readiness for uptake of intention-related material.     

An observation on the spontaneous noticing of prospective memory event-based cues

In event-based prospective memory, current theories make differing predictions as to whether intention-related material can be spontaneously noticed (i.e., noticed without relying on preparatory attentional processes). In 2 experiments, participants formed an intention that was contextually associated to the final phase of the experiment, and lures that overlapped to differing degrees with the features of the intention-related cues were embedded in the initial phase. When participants were outside of the appropriate responding context (i.e., the initial phase), they exhibited slower latencies to lures that exactly matched the features of their intention compared with other types of lures and control words. In addition, on a final remember/know recognition test, participants reported having greater subjective recollection for the occurrence of the exact-match lures. These results suggest that exact-match lures were spontaneously noticed and differentially processed in the absence of any observable preparatory attentional processes. The findings have implications for the theoretical debate over whether preparatory attention must always be relied upon to notice intention-related material.     

Affordance Compatibility Effect for Word Learning in Virtual Reality

Rich sensorimotor interaction facilitates language learning and is presumed to ground conceptual representations. Yet empirical support for early stages of embodied word learning is currently lacking. Finding evidence that sensorimotor interaction shapes learned linguistic representations would provide crucial support for embodied language theories. We developed a gamified word learning experiment in virtual reality in which participants learned the names of six novel objects by grasping and manipulating objects with either their left or right hand. Participants then completed a word–color match task in which they were tested on the same six words and objects. Participants were faster to respond to stimuli in the match task when the response hand was compatible with the hand used to interact with the named object, an effect we refer to as affordance compatibility. In two follow up experiments, we found that merely observing virtual hands interact with the objects was sufficient to acquire a smaller affordance compatibility effect, and we found that the compatibility effect was driven primarily by responses with a compatible hand and not by responses in a compatible spatial location. Our results support theoretical views of language which ground word representations in sensorimotor experiences, and they suggest promising future routes to explore the sensorimotor foundations of higher cognition through immersive virtual experiments.     

Further evidence that object-based correspondence effects are primarily modulated by object location not by grasping affordance

Tipper, Paul and Hayes found object-based correspondence effects for door-handle stimuli for shape judgments but not colour. They reasoned that a grasping affordance is activated when judging dimensions related to a grasping action (shape), but not for other dimensions (colour). Cho and Proctor, however, found the effect with respect to handle position when the bases of the door handles were centred (so handles were positioned left or right; the base-centred condition) but not when the handles were centred (the object-centred condition), suggesting that the effect is driven by object location, not grasping affordance. We conducted an independent replication of Cho and Proctor's design, but with behavioural and event-related potential measures. Participants made shape judgments in Experiment 1 and colour judgments in Experiment 2 on the same door-handle objects. Correspondence effects on response time and errors were obtained in both experiments for the base-centred condition but not the object-centred condition. Effects were absent in the P1 and N1 data, which are consistent with the hypothesis of little binding between visual processing of grasping component and action. These findings question the grasping-affordance view but support a spatial-coding view, suggesting that correspondence effects are modulated primarily by object location.     

On the temporal dynamics of language-mediated vision and vision-mediated language

Recent converging evidence suggests that language and vision interact immediately in non-trivial ways, although the exact nature of this interaction is still unclear. Not only does linguistic information influence visual perception in real-time, but visual information also influences language comprehension in real-time. For example, in visual search tasks, incremental spoken delivery of the target features (e.g., “Is there a red vertical?”) can increase the efficiency of conjunction search because only one feature is heard at a time. Moreover, in spoken word recognition tasks, the visual presence of an object whose name is similar to the word being spoken (e.g., a candle present when instructed to “pick up the candy”) can alter the process of comprehension. Dense sampling methods, such as eye-tracking and reach-tracking, richly illustrate the nature of this interaction, providing a semi-continuous measure of the temporal dynamics of individual behavioral responses. We review a variety of studies that demonstrate how these methods are particularly promising in further elucidating the dynamic competition that takes place between underlying linguistic and visual representations in multimodal contexts, and we conclude with a discussion of the consequences that these findings have for theories of embodied cognition.     

Vision-for-action: The effects of object property discrimination and action state on affordance compatibility effects

When a person views an object, the action the object evokes appears to be activated independently of the person’s intention to act. We demonstrate two further properties of this vision-to-action process. First, it is not completely automatic, but is determined by the stimulus properties of the object that are attended. Thus, when a person discriminates the shape of an object, action affordance effects are observed; but when a person discriminates an object’s color, no affordance effects are observed. The former, shape property is associated with action, such as how an object might be grasped; the latter, color property is irrelevant to action. Second, we also show that the action state of an object influences evoked action. Thus, active objects, with which current action is implied, produce larger affordance effects than passive objects, with which no action is implied. We suggest that the active object activates action simulation processes similar to those proposed in mirror systems.     

Simon-like and functional affordance effects with tools: The effects of object perceptual discrimination and object action state

In the present study two separate stimulus–response compatibility effects (functional affordance and Simon-like effects) were investigated with centrally presented pictures of an object tool (a torch) characterized by a structural separation between the graspable portion and the goal-directed portion. In Experiment 1, participants were required to decide whether the torch was red or blue, while in Experiment 2 they were required to decide whether the torch was upright or inverted. Our results showed that with the same stimulus two types of compatibility effect emerged: one based on the direction signalled by the goal-directed portion of the tool (a Simon-like effect as observed in Experiment 1), and the other based on the actions associated with an object (a functional affordance effect as observed in Experiment 2). Both effects emerged independently of the person's intention to act on the stimulus, but depended on the stimulus properties that were processed in order to perform the task.     

示能性:基于镜像神经元视角的理解

示能性(affordance)概念解释了人的行为与物体功能之间互动、互补的关系。但是,有关示能性生理基础却很少被提及。镜像神经元的发现为示能性生理基础提供了可能的解释。文章根据研究将示能性分为结构示能性和功能示能性,探讨了不同示能性和镜像神经元之间的关系,并提出不同神经通路中的镜像神经元是示能性生理基础的结论。背-背侧分流中的镜像神经元是结构示能性的神经基础,而腹-背侧分流中的镜像神经元是功能示能性的神经基础。     

行动语义、客体背景和判断任务对客体动作承载性的影响

客体动作承载性指人们知觉客体时同时激活的针对客体的行动。当Gibson最初提出动作承载性概念时, 他认为客体的这种属性是可以自动激活的。然而近年来越来越多的证据显示, 在知觉和行动的关系中, 行动语义知识和客体所处背景对客体动作承载性具有调节作用, 表现在人们可以根据这些自上而下的信息选择针对客体的适当的行动。此外, 不同的判断任务在引发客体动作承载性效应上具有特异性, 表现在若判断任务仅需知觉客体表层属性时, 不能激活对客体的动作, 进而不能产生客体动作承载性; 若判断任务需要对客体进行深层加工时, 才会激活对客体的动作。未来研究还需要进一步探究行动语义知识和客体所处背景影响客体动作承载性发生的原因和机制, 以及不同判断任务中出现结果特异性的原因。     

Language-driven anticipatory eye movements in virtual reality

Predictive language processing is often studied by measuring eye movements as participants look at objects on a computer screen while they listen to spoken sentences. This variant of the visual-world paradigm has revealed that information encountered by a listener at a spoken verb can give rise to anticipatory eye movements to a target object, which is taken to indicate that people predict upcoming words. The ecological validity of such findings remains questionable, however, because these computer experiments used two-dimensional stimuli that were mere abstractions of real-world objects. Here we present a visual-world paradigm study in a three-dimensional (3-D) immersive virtual reality environment. Despite significant changes in the stimulus materials and the different mode of stimulus presentation, language-mediated anticipatory eye movements were still observed. These findings thus indicate that people do predict upcoming words during language comprehension in a more naturalistic setting where natural depth cues are preserved. Moreover, the results confirm the feasibility of using eyetracking in rich and multimodal 3-D virtual environments.     

Orthographic facilitation and phonological inhibition in spoken word recognition: A developmental study

We investigated the extent to which learning to read and write affects spoken word recognition. Previous studies have reported orthographic effects on spoken language in skilled readers. However, very few studies have addressed the development of these effects as a function of reading expertise. We therefore studied orthographic neighborhood (ON) and phonological neighborhood (PN) effects in spoken word recognition in beginning and advanced readers and in children with developmental dyslexia. We predicted that whereas both beginning and advanced readers would show normal PN effects, only advanced readers would show ON effects. The results confirmed these predictions. The size of the ON effect on spoken word recognition was strongly predicted by written language experience and proficiency. In contrast, the size of the PN effect was not affected by reading level. Moreover, dyslexic readers showed no orthographic effects on spoken word recognition. In sum, these data suggest that orthographic effects on spoken word recognition are not artifacts of some uncontrolled spoken language property but reflect a genuine influence of orthographic information on spoken word recognition.     

人工智能辅助的自闭症早期患者的筛查与诊断

自闭症谱系障碍(AutisticSpectrumDisorders,ASD)的症状早在婴幼儿期就会显现,越早发现,越早干预,治疗效果越好。传统自闭症早期筛查与诊断在评估方法、流程上存在局限,无法满足大规模筛查和诊断需求。随着人工智能技术的快速发展,使用智能化方法进行自闭症早期大规模无感筛查与诊断逐渐成为可能。近10年间,国内外对自闭症智能化识别方法的探索在经典任务行为、面部表情和情绪、眼动、脑影像、运动控制和运动模式、多模态6个领域积累了丰富的研究成果。未来研究应围绕构建国内自闭症早期智能医学筛查与诊断体系,开发针对婴幼儿患者的筛查工具,构建融合多模态数据的自闭症婴幼儿智能化识别模型,建立结合脑影像技术的自闭症精细化诊断方法等方面来开展。     

Conflict in object affordance revealed by grip force

Viewing objects can result in automatic, partial activation of motor plans associated with them-"object affordance". Here, we recorded grip force simultaneously from both hands in an object affordance task to investigate the effects of conflict between coactivated responses. Participants classified pictures of objects by squeezing force transducers with their left or right hand. Responses were faster on trials where the object afforded an action with the same hand that was required to make the response (congruent trials) compared to the opposite hand (incongruent trials). In addition, conflict between coactivated responses was reduced if it was experienced on the preceding trial, just like Gratton adaptation effects reported in "conflict" tasks (e.g., Eriksen flanker). This finding suggests that object affordance demonstrates conflict effects similar to those shown in other stimulus-response mapping tasks and thus could be integrated into the wider conceptual framework on overlearnt stimulus-response associations. Corrected erroneous responses occurred more frequently when there was conflict between the afforded response and the response required by the task, providing direct evidence that viewing an object activates motor plans appropriate for interacting with that object. Recording continuous grip force, as here, provides a sensitive way to measure coactivated responses in affordance tasks.     

Conflict in object affordance revealed by grip force

Viewing objects can result in automatic, partial activation of motor plans associated with them—“object affordance”. Here, we recorded grip force simultaneously from both hands in an object affordance task to investigate the effects of conflict between coactivated responses. Participants classified pictures of objects by squeezing force transducers with their left or right hand. Responses were faster on trials where the object afforded an action with the same hand that was required to make the response (congruent trials) compared to the opposite hand (incongruent trials). In addition, conflict between coactivated responses was reduced if it was experienced on the preceding trial, just like Gratton adaptation effects reported in “conflict” tasks (e.g., Eriksen flanker). This finding suggests that object affordance demonstrates conflict effects similar to those shown in other stimulus–response mapping tasks and thus could be integrated into the wider conceptual framework on overlearnt stimulus–response associations. Corrected erroneous responses occurred more frequently when there was conflict between the afforded response and the response required by the task, providing direct evidence that viewing an object activates motor plans appropriate for interacting with that object. Recording continuous grip force, as here, provides a sensitive way to measure coactivated responses in affordance tasks.     

Tactile Short-term Memory Revisited

In spite of a large body of empirical research demonstrating the importance of multisensory integration in cognition, there is still little research about multimodal encoding and maintenance effects in working memory. In this study we investigated multimodal encoding in working memory by means of an immediate serial recall task with different modality and format conditions. In a first non-verbal condition participants were presented with sequences of non-verbal inputs representing familiar (concrete) objects, either in visual, auditory or audio-visual formats. In a second verbal condition participants were presented with written, spoken, or bimodally presented words denoting the same objects represented by pictures or sounds in the non-verbal condition. The effects of articulatory suppression were assessed in both conditions. We found a bimodal superiority effect on memory span with non-verbal material, and a larger span with auditory (or bimodal) versus visual presentation with verbal material, with a significant effect of articulatory suppression in the two conditions.     

The representation of delayed intentions: a prospective subject-performed task?

To-be-enacted material is more accessible in tests of recognition and lexical decision than material not intended for action (T. Goschke & J. Kuhl, 1993; R. L. Marsh, J. L. Hicks, & M. L. Bink, 1998). This finding has been attributed to the superior status of intention-related information. The current article explores an alternative (action-superiority) account that draws parallels between the intended enactment effect (IEE) and the subject performed task effect. Using 2 paradigms, the authors observed faster recognition latencies for both enacted and to-be-enacted material. It is crucial to note that there was no evidence of an IEE for items that had already been executed during encoding. The IEE was also eliminated when motor processing was prevented after verbal encoding. These findings suggest an overlap between overt and intended enactment and indicate that motor information may be activated for verbal material in preparation for subsequent execution.     

Separate Pathways for the Processing of Affordance of Neutral and Dangerous Object

Previous studies have shown that observation of object automatically elicits the activation of a reach-to-grasp response specifically directed to interact with the object, which is termed affordance. Recent findings provide evidences that even dangerous objects can evoke aversive affordances. However, these studies only focused on avoidance effect emerged with dangerous objects. It remains unclear whether the neutral object and dangerous object share a common mechanism for the processing of affordance information. The affordance effect is considered to be a type of conflict between the afforded response and the response required by the task. In a symbol judgment task, we simultaneously presented the neutral object and dangerous object to investigate whether the congruency sequence effects (congruency effects following incongruent trials are smaller than those following congruent trials) occurred across the two types of affordance conflict, which is typically assumed that such flexible adjustments can only be observed between the same conflict types. It was found that congruency sequence effects were only generated between the same types of affordance conflict (e.g., neutral object or dangerous object), but not between two different affordance conflict types (e.g., neutral object and dangerous object). These results indicate there may be two different pathways for the processing of affordance of the neutral and dangerous object.     

Eye movements during spoken word recognition in Russian children

This study explores incremental processing in spoken word recognition in Russian 5- and 6-year-olds and adults using free-viewing eye-tracking. Participants viewed scenes containing pictures of four familiar objects and clicked on a target embedded in a spoken instruction. In the cohort condition, two object names shared identical three-phoneme onsets. In the noncohort condition, all object names had unique onsets. Coarse-grain analyses of eye movements indicated that adults produced looks to the competitor on significantly more cohort trials than on noncohort trials, whereas children surprisingly failed to demonstrate cohort competition due to widespread exploratory eye movements across conditions. Fine-grain analyses, in contrast, showed a similar time course of eye movements across children and adults, but with cohort competition lingering more than 1s longer in children. The dissociation between coarse-grain and fine-grain eye movements indicates a need to consider multiple behavioral measures in making developmental comparisons in language processing.