Interactions of reading and semantics along the ventral visual processing stream

Converging evidence supports a distributed-plus-hub view of semantic processing, in which there are distributed modular semantic sub-systems (e.g., for shape, colour, and action) connected to an amodal semantic hub. Furthermore, object semantic processing of colour and shape, and lexical reading and identification, are processed mainly along the ventral stream, while action semantic processing occurs mainly along the dorsal stream. In Experiment 1, participants read a prime word that required imagining either the object or action referent, and then named a lexical word target. In Experiments 2 and 3, participants performed a lexical decision task (LDT) with the same targets as in Experiment 1, in the presence of foils that were legal nonwords (NW; Experiment 2) or pseudohomophones (PH; Experiment 3). Semantic priming was similar in effect size regardless of prime type for naming, but was greater for object primes than action primes for the LDT with PH foils, suggesting a shared-stream advantage when the task demands focus on orthographic lexical processing. These experiments extend the distributed-plus-hub model, and provide a novel paradigm for further research.     

Short-term action intentions overrule long-term semantic knowledge

In the present study, we investigated whether the preparation of an unusual action with an object (e.g. bringing a cup towards the eye) could selectively overrule long-term semantic representations. In the first experiment it was found that unusual action intentions activated short-term semantic goal representations, rather than long-term conceptual associations. In a second experiment the reversal of long-term priming effects was replicated, while reducing the need for internal verbalization as a possible strategy to accomplish the task. Priming effects in the first two experiments were found to involve the selection of object knowledge at a semantic level, rather than reflecting a general effect of action preparation on word processing (Experiment 3). Finally, in a fourth experiment short-term priming effects were shown to extend beyond a lexical level by showing faster responses to pictures representing the short-term action goal. Together, the present findings extend the ‘selection-for-action’ principle previously used in visual attention to a semantic level, by showing that semantic information is selectively activated in line with the short-term goal of the actor.     

Grasping objects by their handles: a necessary interaction between cognition and action

Research has illustrated dissociations between "cognitive" and "action" systems, suggesting that different representations may underlie phenomenal experience and visuomotor behavior. However, these systems also interact. The present studies show a necessary interaction when semantic processing of an object is required for an appropriate action. Experiment 1 demonstrated that a semantic task interfered with grasping objects appropriately by their handles, but a visuospatial task did not. Experiment 2 assessed performance on a visuomotor task that had no semantic component and showed a reversal of the effects of the concurrent tasks. In Experiment 3, variations on concurrent word tasks suggested that retrieval of semantic information was necessary for appropriate grasping. In all, without semantic processing, the visuomotor system can direct the effective grasp of an object, but not in a manner that is appropriate for its use.     

Controlling the unconscious: attentional task sets modulate subliminal semantic and visuomotor processes differentially

Are unconscious processes susceptible to attentional influences? In two subliminal priming experiments, we investigated whether task sets differentially modulate the sensitivity of unconscious processing pathways. We developed a novel procedure for masked semantic priming of words (Experiment 1) and masked visuomotor priming of geometrical shapes (Experiment 2). Before presentation of the masked prime, participants performed an induction task in which they attended to either semantic or perceptual object features designed to activate a semantic or perceptual task set, respectively. Behavioral and electrophysiological effects showed that the induction tasks differentially modulated subliminal priming: Semantic priming, which involves access to conceptual meaning, was found after the semantic induction task but not after the perceptual induction task. Visuomotor priming was observed after the perceptual induction task but not after the semantic induction task. These results demonstrate that unconscious cognition is influenced by attentional control. Unconscious processes in perceptual and semantic processing streams are coordinated congruently with higher-level action goals.     

See what I mean? An ERP study of the effect of background knowledge on novel object processing

Two event-related potential (ERP) experiments were used to examine the representation of object feature information and background knowledge in semantic memory. Participants were trained on novel object categories with three features and were tested with new exemplars that were complete or were missing one to two features that were essential or nonessential to object function. In both a category membership judgment task (Experiment 1) and a parts detection task (Experiment 2), the N400, a functionally specific measure of semantic access, was graded with feature number but was insensitive to knowledge-based feature importance. A separable ERP effect related to knowledge was seen in Experiment 1 as an enhanced frontocentral negativity (beginning ∼300 msec) to exemplars missing a nonessential versus an essential feature, but this effect did not manifest when background knowledge was less task relevant (Experiment 2). Thus, similarity- and knowledge-based effects are separable, and the locus of knowledge effects varies with task demands but does not seem to arise from facilitated semantic access.     

Attentional selection during preparation of prehension movements

In two experiments coupling between dorsal attentional selection for action and ventral attentional selection for perception during preparation of prehension movements was examined. In a dual-task paradigm subjects had to grasp an X-shaped object with either the left or the right hand's thumb and index finger. Simultaneously a discrimination task was used to measure visual attention prior to the execution of the prehension movements: Mask items transiently changed into distractors or discrimination targets. There was exactly one discrimination target per trial, which appeared at one of the four branch ends of the object. In Experiment 1 target position varied randomly while in Experiment 2 it was constant and known to subjects in each block of trials. In both experiments discrimination performance was significantly better for discrimination target positions at to-be-grasped branch ends than for not-to-be-grasped branch ends. We conclude that during preparation of prehension movements visual attention is largely confined to those parts of an object that will be grasped.     

The neural substrates of action retrieval: An examination of semantic and visual routes to action

We report three PET experiments that examine the neural substrates of the conceptual components of action retrieval. In all three experiments, subjects made action or screen-size decisions to familiar objects presented either as pictures or written words (the names ofthe objects). In Experiment 1, a third task was included, requiring a decision on the real-life size of the stimuli. In Experiment 2, a third stimulus type was included, with action and size decisions also performed on pictures of meaningless novel objects. Finally, in Experiment 3, we changed the response mode from a button press to a more explicit movement made with a “manipulandum”. Based on neuropsychological findings, we predicted that when action responses were made to pictures of familiar or novel objects, relative to words, there would be less activation in semantic regions but greater activation in visual, motor, and perhaps parietal cortices. We found that, action relative to screen-size decisions on both pictures and words activated the left hemisphere temporo-frontal semantic system with activation in the left posterior middle temporal cortex specific to action retrieval (Experiment 1). In addition, action retrieval elicited more activation for (1) words than pictures in areas associated with semantics; and (2) novel objects than words or familiar objects in areas associated with pre-semantic object processing. These results are discussed in the context of semantic and visual routes to action retrieval.     

Perceptual priming is not a necessary consequence of semantic classification of pictures

Four experiments investigated how repetition priming of object recognition is affected by the task performed in the prime and test phases. In Experiment 1 object recognition was tested using both vocal naming and two different semantic decision tasks (whether or not objects were manufactured, and whether or not they would be found inside the house). Some aspects of the data were inconsistent with contemporary models of object recognition. Specifically, object priming was eliminated with some combinations of prime and test tasks, and there was no evidence of perceptual (as opposed to conceptual or response) priming in either semantic classification task, even though perceptual identification of the objects is required for at least one of these tasks. Experiment 2 showed that even when perceptual demands were increased by brief presentation, the inside task showed no perceptual priming. Experiment 3 showed that the inside task did not appear to be based on conceptual priming either, as it was not primed significantly when the prime decisions were made to object labels. Experiment 4 showed that visual sensitivity could be restored to the inside task following practice on the task, supporting the suggestion that a critical factor is whether the semantic category is preformed or must be computed. The results show that the visual representational processes revealed by object priming depend crucially on the task chosen.     

Creating semantics in tool use

This article presents the first evidence for a functional link between tool use and the processing of abstract symbols like Arabic numbers. Participants were required to perform a tool-use task after the processing of an Arabic number. These numbers represented either a small (2 or 3) or a large magnitude (8 or 9). The tool-use task consisted in using inverse pliers for gripping either a small or a large object. The inverse pliers enable to dissociate the hand action from the tool action in relation to the object (i.e., closing the hand led to an opening of the tool and vice versa). The number/tool hypothesis predicts that the quantity representation associated with Arabic numbers will interact with the action of the tool toward the object. Conversely, the number/hand hypothesis predicts that the quantity associated with numbers will interact with the action of the hand toward the tool. Results confirmed the first hypothesis and rejected the second. Indeed, large numbers interacted with the action of the tool, such that participants were longer to perform an “opening-hand/closing-tool” action after the processing of large numbers. Moreover, no effect was detected for small numbers, confirming previous studies which used only finger movements. Altogether, our finding suggests that the well-known finger/number interaction can be reversed with tool use.     

Imitative response tendencies following observation of intransitive actions

Clear and unequivocal evidence shows that observation of object affordances or transitive actions facilitates the activation of a compatible response. By contrast, the evidence showing response facilitation following observation of intransitive actions is less conclusive because automatic imitation and spatial compatibility have been confounded. Three experiments tested whether observation of a finger movement (i.e., an intransitive action) in a choice reaction-time task facilitates the corresponding finger movement response because of imitation, a common spatial code, or some combination of both factors. The priming effects of a spatial and an imitative stimulus were tested in combination (Experiment 1), in opposition (Experiment 2), and independently (Experiment 3). Contrary to previous findings, the evidence revealed significant contributions from both automatic imitation and spatial compatibility, but the priming effects from an automatic tendency to imitate declined significantly across a block of trials whereas the effects of spatial compatibility remained constant or increased slightly. These differential effects suggest that priming associated with automatic imitation is mediated by a different regime than priming associated with spatial compatibility.     

Semantic activation and letter search: Blocking or suppression?

Two experiments investigated associative priming with a letter-search prime task where either the prime and letter probe were presented simultaneously, or the letter probe appeared 200 ms (Experiment 1) or 300 ms (Experiment 2) after the prime. Weak associative priming was observed in both experiments, but unlike Stolz and Besner (1996) we found no evidence that priming was increased when the probe was delayed. However, strong associative priming was observed when a semantic decision had to be made on the prime (Experiment 3). Our results are consistent with an account where semantic activation of the prime occurs but its action on the target is suppressed by the prime task. The persistence of weak priming effects with the letter search task is explained in terms of the low-frequency items used.     

Mental object rotation and the planning of hand movements

Recently, we showed that the simultaneous execution of rotational hand movements interferes with mental object rotation, provided that the axes of rotation coincide in space. We hypothesized that mental object rotation and the programming of rotational hand movements share a common process presumably involved in action planning. Two experiments are reported here that show that the mere planning of a rotational hand movement is sufficient to cause interference with mental object rotation. Subjects had to plan different spatially directed hand movements that they were asked to execute only after they had solved a mental object rotation task. Experiment 1 showed that mental object rotation was slower if hand movements were planned in a direction opposite to the presumed mental rotation direction, but only if the axes of hand rotation and mental object rotation were parallel in space. Experiment 2 showed that this interference occurred independent of the preparatory hand movements observed in Experiment 1. Thus, it is the planning of hand movements and not their preparation or execution that interferes with mental object rotation. This finding underlines the idea that mental object rotation is an imagined (covert) action, rather than a pure visual-spatial imagery task, and that the interference between mental object rotation and rotational hand movements is an interference between goals of actions.     

Mental object rotation and the planning of hand movements.

Recently, we showed that the simultaneous execution of rotational hand movements interferes with mental object rotation, provided that the axes of rotation coincide in space. We hypothesized that mental object rotation and the programming of rotational hand movements share a common process presumably involved in action planning. Two experiments are reported here that show that the mere planning of a rotational hand movement is sufficient to cause interference with mental object rotation. Subjects had to plan different spatially directed hand movements that they were asked to execute only after they had solved a mental object rotation task. Experiment 1 showed that mental object rotation was slower if hand movements were planned in a direction opposite to the presumed mental rotation direction, but only if the axes of hand rotation and mental object rotation were parallel in space. Experiment 2 showed that this interference occurred independent of the preparatory hand movements observed in Experiment 1. Thus, it is the planning of hand movements and not their preparation or execution that interferes with mental object rotation. This finding underlines the idea that mental object rotation is an imagined (covert) action, rather than a pure visual-spatial imagery task, and that the interference between mental object rotation and rotational hand movements is an interference between goals of actions.     

汉语词汇习得的年龄效应：语义假设的证据

通过三个实验考察了词汇加工中的习得年龄效应。实验一对比了汉字命名和图片命名中词汇习得的年龄效应,发现图片命名中存在着较大的词汇习得的年龄效应,汉字命名中没有出现这种效应。实验二和实验三分别采用语义范畴判断任务和图片语义分类任务,两个实验均发现了词汇习得的年龄效应。实验结果说明,词汇习得的年龄效应至少部分来源于语义加工的层次,结果支持了语义假设的观点     

What’s in an object file? Evidence from priming studies

Six experiments were conducted to investigate the nature of the contents of object files, temporary representations that store information about objects. Experiment 1 used a lexical priming paradigm with a lexical decision task, in which the prime and target could appear in either the same or different locations. The results indicated a greater priming effect when the prime and target appeared in the same location than when they appeared in different locations (object- or location-specific priming). Experiment 2 replicated these findings for objects that changed position during the display. Experiment 3 demonstrated that these findings reflected the inclusion of abstract identity information, rather than physical form, in object files. Three additional experiments tested for the presence of three types of semantic information (related concepts, semantic features, and category membership) in object files. No object-specific priming effects were found. Taken together, these experiments suggest that an object file includes identity information, but not semantic information. Implications of the results for object file theory are discussed.     

Colour specificity in episodic and in perceptual object recognition with enhanced colour impact

In three experiments we investigated the perceptual specificity of explicit (old-new object recognition) and implicit memory (word-picture matching) for colour. In order to enhance the impact of colour on processing, we increased the number of colours per object and we impaired shape information. We presented multicoloured pictures (Experiment 1), blurred and partially occluded pictures (Experiment 2) and coloured line drawings in visual noise (Experiment 3). Experiments 1 and 2 had an intentional study phase; the study phase of Experiment 3 was an incidental colour or category naming task. Changing colour from study to test always had negative effects on episodic recognition although colour was irrelevant. In contrast, in the matching task old pictures were generally matched faster than new ones independent of their colour congruence. In Experiment 3, an additional small advantage of congruent colours and of semantic processing occurred. We conclude that two different memory representations contribute to these tasks. Changes of an achromatic, more abstract representation, that is used in normal object recognition, and a representation of the specific exemplar that includes colour. This latter token is used in episodic recognition as well as in unusual perceptual tasks (Experiment 3).     

Beyond words: evidence for automatic language–gesture integration of symbolic gestures but not dynamic landscapes

Understanding actions based on either language or action observation is presumed to involve the motor system, reflecting the engagement of an embodied conceptual network. We examined how linguistic and gestural information were integrated in a series of cross-domain priming studies. We varied the task demands across three experiments in which symbolic gestures served as primes for verbal targets. Primes were clips of symbolic gestures taken from a rich set of emblems. Participants responded by making a lexical decision to the target (Experiment 1), naming the target (Experiment 2), or performing a semantic relatedness judgment (Experiment 3). The magnitude of semantic priming was larger in the relatedness judgment and lexical decision tasks compared to the naming task. Priming was also observed in a control task in which the primes were pictures of landscapes with conceptually related verbal targets. However, for these stimuli, the amount of priming was similar across the three tasks. We propose that action observation triggers an automatic, pre-lexical spread of activation, consistent with the idea that language–gesture integration occurs in an obligatory and automatic fashion.     

A motor isolation effect: When object manipulability modulates recall performance

Previous studies suggested that the language production architecture is recruited during verbal retention, and others proposed that spatial memory relies on the oculomotor system. The objective of the present study was to investigate the role of the motor system in object memory, by examining the effect of objects' affordances on retention. In a serial recall task, we manipulated the manipulability of objects to retain in memory. We used an isolation paradigm where we isolated the manipulability level of one object from the list. We showed that recall performance improved for the isolated object (Experiment 1) and that this advantage was abolished when participants were required to perform motor suppression during the task (Experiment 2). In Experiment 3, we showed that the abolition of the motor isolation effect in Experiment 2 was not due to an effect of distraction since motor suppression was shown not to interfere with a semantic isolation effect. It is argued that motor affordances play a role in object memory, but only when the motor characteristics of an object allow discriminating it from the other objects in the list.     

Embodied grounding of memory: Toward the effects of motor execution on memory consolidation

Behavioural and neuroscientific research has provided evidence for a strong functional link between the neural motor system and lexical–semantic processing of action-related language. It remains unclear, however, whether the impact of motor actions is restricted to online language comprehension or whether sensorimotor codes are also important in the formation and consolidation of persisting memory representations of the word's referents. The current study now demonstrates that recognition performance for action words is modulated by motor actions performed during the retention interval. Specifically, participants were required to learn words denoting objects that were associated with either a pressing or a twisting action (e.g., piano, screwdriver) and words that were not associated to actions. During a 6–8-minute retention phase, participants performed an intervening task that required the execution of pressing or twisting responses. A subsequent recognition task revealed a better memory for words that denoted objects for which the functional use was congruent with the action performed during the retention interval (e.g., pepper mill–twisting action, doorbell–pressing action) than for words that denoted objects for which the functional use was incongruent. In further experiments, we were able to generalize this effect of selective memory enhancement of words by performing congruent motor actions to an implicit perceptual (Experiment 2) and implicit semantic memory test (Experiment 3). Our findings suggest that a reactivation of motor codes affects the process of memory consolidation and emphasizes therefore the important role of sensorimotor codes in establishing enduring semantic representations.