When motor congruency modulates immediate memory for objects

In line with the embodied cognition view, some researchers have suggested that our capacity to retain information relies on the perceptual and motor systems used to interact with our environment (Barsalou, 1999; Glenberg, 1997). For instance, the language production architecture would be responsible for the retention of verbal materials such as a list of words (Acheson & MacDonald, 2009). However, evidence for the role of the motor system in object memory is still limited. In the present experiments, participants were asked to retain lists of objects in memory. During encoding, participants had to pantomime an action to grasp (Experiments 1A & 1B) or to use the objects (Experiment 2) that was either congruent or incongruent with the objects to be retained. The results showed that performing an incongruent action impaired memory performance compared to a congruent action. This suggests that motor affordances play a role during object retention. The results are discussed in light of the embodied cognition view.     

Motor affordance and its role for visual working memory: evidence from fMRI studies

We examined the role of motor affordances of objects for working memory retention processes. Three experiments are reported in which participants passively viewed pictures of real world objects or had to retain the objects in working memory for a comparison with an S2 stimulus. Brain activation was recorded by means of functional magnetic resonance imaging (fMRI). Retaining information about objects for which hand actions could easily be retrieved (manipulable objects) in working memory activated the hand region of the ventral premotor cortex (PMC) contralateral to the dominant hand. Conversely, nonmanipulable objects activated the left inferior frontal gyrus. This suggests that working memory for objects with motor affordance is based on motor programs associated with their use. An additional study revealed that motor program activation can be modulated by task demands: Holding manipulable objects in working memory for an upcoming motor comparison task was associated with left ventral PMC activation. However, retaining the same objects for a subsequent size comparison task led to activation in posterior brain regions. This suggests that the activation of hand motor programs are under top down control. By this they can flexibly be adapted to various task demands. It is argued that hand motor programs may serve a similar working memory function as speech motor programs for verbalizable working memory contents, and that the premotor system mediates the temporal integration of motor representations with other task-relevant representations in support of goal oriented behavior.     

Effects of motor congruence on visual working memory

Grounded-cognition theories suggest that memory shares processing resources with perception and action. The motor system could be used to help memorize visual objects. In two experiments, we tested the hypothesis that people use motor affordances to maintain object representations in working memory. Participants performed a working memory task on photographs of manipulable and nonmanipulable objects. The manipulable objects were objects that required either a precision grip (i.e., small items) or a power grip (i.e., large items) to use. A concurrent motor task that could be congruent or incongruent with the manipulable objects caused no difference in working memory performance relative to nonmanipulable objects. Moreover, the precision- or power-grip motor task did not affect memory performance on small and large items differently. These findings suggest that the motor system plays no part in visual working memory.     

Semantic memory disorders

Semantic memory encompasses knowledge of objects, facts and words. A number of brain regions are probably involved, but the left infero-lateral temporal lobe appears to play a key role. The separability of semantic memory from episodic (or autobiographical) memory is a focus of current debate. Impaired semantic memory is a common feature of Alzheimer's disease but is invariably overshadowed by a profound deficit in episodic memory. In semantic dementia, a rarer disorder associated with focal temporal-lobe atrophy, there is selective loss of semantic memory, characterized by preservation of superordinate knowledge of words, and objects, but loss of finer-grained information. This pattern can be interpreted as a degradation of features from a distributed network of semantic representations. Following Herpes simplex encephalitis, patients sometimes show disproportionate loss of knowledge for natural kinds (e.g. animals) with relative preservation of knowledge about artefacts, this may reflect differential damage to neural systems critical for perceptual as opposed to functional features, perceptual properties being more salient in knowledge about natural kinds.     

Cross-modal working memory binding and L1-L2 word learning

The ability to create temporary binding representations of information from different sources in working memory has recently been found to relate to the development of monolingual word recognition in children. The current study explored this possible relationship in an adult word-learning context. We assessed whether the relationship between cross-modal working memory binding and lexical development would be observed in the learning of associations between unfamiliar spoken words and their semantic referents, and whether it would vary across experimental conditions in first- and second-language word learning. A group of English monolinguals were recruited to learn 24 spoken disyllable Mandarin Chinese words in association with either familiar or novel objects as semantic referents. They also took a working memory task in which their ability to temporarily bind auditory-verbal and visual information was measured. Participants’ performance on this task was uniquely linked to their learning and retention of words for both novel objects and for familiar objects. This suggests that, at least for spoken language, cross-modal working memory binding might play a similar role in second language-like (i.e., learning new words for familiar objects) and in more native-like situations (i.e., learning new words for novel objects). Our findings provide new evidence for the role of cross-modal working memory binding in L1 word learning and further indicate that early stages of picture-based word learning in L2 might rely on similar cognitive processes as in L1.     

Privileged access to action for objects relative to words

We compared action (pour or twist?) and contextual/semantic (found in kitchen?) decisions made to pictures of objects, nonobjects, and words. Although there was no advantage for objects over words in contextual/semantic decisions, there was an advantage for objects over words and nonobjects in action decisions. For objects, both action and contextual/semantic decisions were faster than naming; for words, the opposite occurred. These results extend the early results of Potter and Faulconer (1975) that there is privileged access to semantic memory for objects relative to that for words and privileged access to phonology for words. Our data suggest that, for objects, there is privileged access to action knowledge rather than to all forms of semantic knowledge and that this is contingent on learned associations between objects and actions.     

The selective impairment of semantic memory

The selective impairment of semantic memory is described in three patients with diffuse cerebrallesions. These patients, selected on the basis of a failure to recognize or identify common objects (agnosia for objects), were investigated in detail. In particular, their perceptual, language and memory functions were assessed, and the limits and properties of their recognition difficulties explored.

It was found that knowledge of pictorial representations of objects, and of words, was impaired or impoverished, and in both instances knowledge of subordinate categories was more vulnerable than superordinate categories. Evidence is presented that this impairment of semantic memory cannot be accounted for by intellectual impairment, sensory or perceptual deficits, or expressive language disorder. The implications of damage to the semantic memory system for the operation of other cognitive systems, in particular short and long-term memory functions, are considered. Some tentative evidence for the structural basis for a hierarchically organized modality-specific semantic memory system is discussed.     

Semantic short-term memory and its role in sentence processing: a replication

Previous studies have shown that an aphasic patient (AB) with a semantic short-term memory deficit (STM) had difficulties comprehending and producing sentences with structures that demanded the simultaneous retention of several individual word meanings (Martin & Freedman, 2001a, 2001b; Martin & Romani, 1994; Martin, Shelton, & Yaffee, 1994). The present study provides a replication of these findings with an additional case (ML) who shows a striking dissociation between preserved semantic knowledge and disrupted semantic STM. ML performed poorly on comprehension tasks for sentences that required the retention of single word meanings across several intervening words. In contrast, he did not show an effect of intervening words on processing grammatical relations. ML had difficulty producing adjective-noun phrases, though able to produce the individual nouns and adjectives. These findings support the contention that there is a semantic retention capacity, involved in both comprehension and production, that is separate from semantic knowledge representations.     

Binding actions and scenes in visual long-term memory

How does visual long-term memory store representations of different entities (e.g., objects, actions, and scenes) that are present in the same visual event? Are the different entities stored as an integrated representation in memory, or are they stored separately? To address this question, we asked observers to view a large number of events; in each event, an action was performed within a scene. Afterward, the participants were shown pairs of action–scene sets and indicated which of the two they had seen. When the task required recognizing the individual actions and scenes, performance was high (80 %). Conversely, when the task required remembering which actions had occurred within which scenes, performance was significantly lower (59 %). We observed this dissociation between memory for individual entities and memory for entity bindings across multiple testing conditions and presentation durations. These experiments indicate that visual long-term memory stores information about actions and information about scenes separately from one another, even when an action and scene were observed together in the same visual event. These findings also highlight an important limitation of human memory: Situations that require remembering actions and scenes as integrated events (e.g., eyewitness testimony) may be particularly vulnerable to memory errors.     

Linking somatic and symbolic representation in semantic memory: the dynamic multilevel reactivation framework

Biological plausibility is an essential constraint for any viable model of semantic memory. Yet, we have only the most rudimentary understanding of how the human brain conducts abstract symbolic transformations that underlie word and object meaning. Neuroscience has evolved a sophisticated arsenal of techniques for elucidating the architecture of conceptual representation. Nevertheless, theoretical convergence remains elusive. Here we describe several contrastive approaches to the organization of semantic knowledge, and in turn we offer our own perspective on two recurring questions in semantic memory research: (1) to what extent are conceptual representations mediated by sensorimotor knowledge (i.e., to what degree is semantic memory embodied)? (2) How might an embodied semantic system represent abstract concepts such as modularity, symbol, or proposition? To address these questions, we review the merits of sensorimotor (i.e., embodied) and amodal (i.e., disembodied) semantic theories and address the neurobiological constraints underlying each. We conclude that the shortcomings of both perspectives in their extreme forms necessitate a hybrid middle ground. We accordingly propose the Dynamic Multilevel Reactivation Framework—an integrative model predicated upon flexible interplay between sensorimotor and amodal symbolic representations mediated by multiple cortical hubs. We discuss applications of the dynamic multilevel reactivation framework to abstract and concrete concept representation and describe how a multidimensional conceptual topography based on emotion, sensation, and magnitude can successfully frame a semantic space containing meanings for both abstract and concrete words. The consideration of ‘abstract conceptual features’ does not diminish the role of logical and/or executive processing in activating, manipulating and using information stored in conceptual representations. Rather, it proposes that the materials upon which these processes operate necessarily combine pure sensorimotor information and higher-order cognitive dimensions involved in symbolic representation.     

Motor adaptation and manual transfer: Insight into the persistent nature of sensorimotor representations

It is well known that sensorimotor memories are built and updated through experience with objects. These representations are useful to anticipatory and feedforward control processes that preset grip and load forces during lifting. When individuals lift objects with qualities that are not congruent with their memory-derived expectations, feedback processes adjust motor plans to achieve successful lifts and contribute to the updating of the stored representations. The two experiments presented examine motor adaptation to an illusory size–weight lifting task, and the transfer of this motor adaptation to the unexposed hand. In Experiment 1, performers acquired motor adaptation with their right hand and transfer was measured on their left hand. In Experiment 2, adaptation was acquired with the left hand and transfer was measured on the right hand. In order to investigate the persistence of sensorimotor memories, these experiments measure adaptation, retention, and transfer after 15 min and 24 h delay periods. Both experiments confirm that experience with objects leads to adaptation of force scaling processes, that these adaptations transcend effector and are persistent. The results are discussed in terms favouring interpretations that describe motor adaptations to illusion as being centrally available.     

Motor action and emotional memory

Can simple motor actions affect how efficiently people retrieve emotional memories, and influence what they choose to remember? In Experiment 1, participants were prompted to retell autobiographical memories with either positive or negative valence, while moving marbles either upward or downward. They retrieved memories faster when the direction of movement was congruent with the valence of the memory (upward for positive, downward for negative memories). Given neutral-valence prompts in Experiment 2, participants retrieved more positive memories when instructed to move marbles up, and more negative memories when instructed to move them down, demonstrating a causal link from motion to emotion. Results suggest that positive and negative life experiences are implicitly associated with schematic representations of upward and downward motion, consistent with theories of metaphorical mental representation. Beyond influencing the efficiency of memory retrieval, the direction of irrelevant, repetitive motor actions can also partly determine the emotional content of the memories people retrieve: moving marbles upward (an ostensibly meaningless action) can cause people to think more positive thoughts.     

熟练粤语-普通话双言者听觉词的语言表征*

以熟练粤语-普通话双言者为被试, 考察粤语-普通话双言者对听觉词的语言表征。实验1呈现粤语和普通话同形异音的词, 实验2呈现粤语和普通话异形异音的词, 要求被试做语义决定。结果发现, 粤语-普通话双言者对听觉词的语言表征类似于双语者, 即普通话和粤语的语义共同表征, 词汇分别表征。所以如此, 是由粤语的特点和它与普通话的关系决定的。     

Usage of semantic representations in recognition memory

Meanings of words facilitate false acceptance as well as correct rejection of lures in recognition memory tests, depending on the experimental context. This suggests that semantic representations are both directly and indirectly (i.e., mediated by perceptual representations) used in remembering. Studies using memory conjunction errors (MCEs) paradigms, in which the lures consist of component parts of studied words, have reported semantic facilitation of rejection of the lures. However, attending to components of the lures could potentially cause this. Therefore, we investigated whether semantic overlap of lures facilitates MCEs using Japanese Kanji words in which a whole-word image is more concerned in reading. Experiments demonstrated semantic facilitation of MCEs in a delayed recognition test (Experiment 1), and in immediate recognition tests in which participants were prevented from using phonological or orthographic representations (Experiment 2), and the salient effect on individuals with high semantic memory capacities (Experiment 3). Additionally, analysis of the receiver operating characteristic suggested that this effect is attributed to familiarity-based memory judgement and phantom recollection. These findings indicate that semantic representations can be directly used in remembering, even when perceptual representations of studied words are available.     

Sleep enhances explicit recollection in recognition memory

Recognition memory is considered to be supported by two different memory processes, i.e., the explicit recollection of information about a previous event and an implicit process of recognition based on an acontextual sense of familiarity. Both types of memory supposedly rely on distinct memory systems. Sleep is known to enhance the consolidation of memories, with the different sleep stages affecting different types of memory. In the present study, we used the process-dissociation procedure to compare the effects of sleep on estimates of explicit (recollection) and implicit (familiarity) memory formation on a word-list discrimination task. Subjects studied two lists of words before a 3-h retention interval of sleep or wakefulness, and recognition was tested afterward. The retention intervals were positioned either in the early night when sleep is dominated by slow-wave sleep (SWS), or in the late night, when sleep is dominated by REM sleep. Sleep enhanced explicit recognition memory, as compared with wakefulness (P < 0.05), whereas familiarity was not affected by sleep. Moreover, explicit recognition was particularly enhanced after sleep in the early-night retention interval, and especially when the words were presented with the same contextual features as during learning, i.e., in the same font (P < 0.05). The data indicate that in a task that allows separating the contribution of explicit and implicit memory, sleep particularly supports explicit memory formation. The mechanism of this effect appears to be linked to SWS.     

The sensorimotor contributions to implicit memory, familiarity, and recollection

The sensorimotor contributions to memory for prior occurrence were investigated. Previous research has shown that both implicit memory and familiarity draw on gains in stimulus-related processing fluency for old, compared with novel, stimuli, but recollection does not. Recently, it has been demonstrated that processing fluency itself resides in stimulus-specific motor simulations or reenactment (e.g., covert pronouncing simulations for words as stimuli). Combining these lines of evidence, it was predicted that stimulus-specific motor interference preventing simulations should impair both implicit memory and familiarity but leave recollection unaffected. This was tested for words as verbal stimuli associated to pronouncing simulations in the oral muscle system (but also for tunes as vocal stimuli and their associated vocal system, Experiment 2). It was found that oral (e.g., chewing gum), compared with manual (kneading a ball), motor interference prevented mere exposure effects (Experiments 1-2), substantially reduced repetition priming in word fragment completion (Experiment 3), reduced the familiarity estimates in a remember-know task (Experiment 5) and in receiver-operating characteristics (Experiment 6), and completely neutralized familiarity measured by self-reports (Experiment 4) and skin conductance responses (Experiment 7), while leaving recollection and free recall unaffected (across Experiments 1-7). This pattern establishes a rare memory dissociation in healthy participants, that is, explicit without implicit memory or recognizing without feeling familiar. Implications for embodied memory and neuropsychology are discussed.     

Involuntary aware memory enhances priming on a conceptual implicit memory task

This study investigated the role of involuntary aware memory (phenomenological awareness of the study episode) on a conceptual implicit memory task. Subjects studied words under levels of processing conditions (LOP: nonsemantic vs. semantic) and then received a category exemplar generation priming task. Subjects were either aware or unaware of the study test relationship. Subjects who were study test aware showed more priming for semantic study than subjects who were study test unaware, thus producing an LOP dissociation between them. The findings suggest that involuntary aware memory can enhance performance on conceptual implicit memory tasks, thus having theoretical implications for implicit and explicit memory.     

Multiple systems of spatial memory and action

Recent findings from spatial cognition and cognitive neuroscience suggest that different types of mental representations could mediate the off-line retrieval of spatial relations from memory and the on-line guidance of motor actions in space. As a result, a number of models proposing multiple systems of spatial memory have been recently formulated. In the present article we review these models and we evaluate their postulates based on available experimental evidence. Furthermore, we discuss how a multiple-system model can apply to situations in which people reason about their immediate surroundings or non-immediate environments by incorporating a model of sensorimotor facilitation/interference. This model draws heavily on previous accounts of sensorimotor interference and takes into account findings from the stimulus-response compatibility literature.     

Implicit associative responses influence encoding in memory

In list-learning experiments, the orienting question asked of one item may influence the processing of, and memory for, later items in the list. Four experiments demonstrated that words that are not related to their own orienting question, but that are semantically related to the question asked of a previous item, are better recalled than are words that are not related to any question in the list. Factors that influence this memory enhancement include the number of times relevant orienting questions appear during study, as well as the retention interval. Experiment 4 revealed that this effect is contingent upon conscious awareness: Item-category relations do not enhance recall if the subject is not aware of them. The results of the four experiments imply that semantic categories can be primed as implicit associative responses to words, which influence memory.     

Eighteen-month-olds integrate verbal cues into their action processing: Evidence from ERPs and mu power

Behavioral research has shown that infants use both behavioral cues and verbal cues when processing the goals of others’ actions. For instance, 18-month-olds selectively imitate an observed goal-directed action depending on its (in)congruence with a model’s previous verbal announcement of a desired action goal. This EEG-study analyzed the electrophysiological underpinnings of these behavioral findings on the two functional levels of conceptual action processing and motor activation. Mid-latency mean negative ERP amplitude and mu-frequency band power were analyzed while 18-month-olds (N = 38) watched videos of an adult who performed one out of two potential actions on a novel object. In a within-subjects design, the action demonstration was preceded by either a congruent or an incongruent verbally announced action goal (e.g., “up” or “down” and upward movement). Overall, ERP negativity did not differ between conditions, but a closer inspection revealed that in two subgroups, about half of the infants showed a broadly distributed increased mid-latency ERP negativity (indicating enhanced conceptual action processing) for either the congruent or the incongruent stimuli, respectively. As expected, mu power at sensorimotor sites was reduced (indicating enhanced motor activation) for congruent relative to incongruent stimuli in the entire sample. Both EEG correlates were related to infants’ language skills. Hence, 18-month-olds integrate action-goal-related verbal cues into their processing of others’ actions, at the functional levels of both conceptual processing and motor activation. Further, cue integration when inferring others’ action goals is related to infants’ language proficiency.