Abstract versus modality-specific memory representations in processing auditory and visual speech

Serial recall of lip-read, auditory, and audiovisual memory lists with and without a verbal suffix was examined. Recency effects were the same in the three presentation modalities. The disrupting effect of a suffix was largest when it was presented in the same modality as the list items. The results suggest that abstract linguistic as well as modality-specific codes play a role in memory for auditory and visual speech.     

The meaning of ‘life’ and other abstract words: Insights from neuropsychology

There are a number of long‐standing theories on how the cognitive processing of abstract words, like ‘life’, differs from that of concrete words, like ‘knife’. This review considers current perspectives on this debate, focusing particularly on insights obtained from patients with language disorders and integrating these with evidence from functional neuroimaging studies. The evidence supports three distinct and mutually compatible hypotheses. (1) Concrete and abstract words differ in their representational substrates, with concrete words depending particularly on sensory experiences and abstract words on linguistic, emotional, and magnitude‐based information. Differential dependence on visual versus verbal experience is supported by the evidence for graded specialization in the anterior temporal lobes for concrete versus abstract words. In addition, concrete words have richer representations, in line with better processing of these words in most aphasic patients and, in particular, patients with semantic dementia. (2) Abstract words place greater demands on executive regulation processes because they have variable meanings that change with context. This theory explains abstract word impairments in patients with semantic‐executive deficits and is supported by neuroimaging studies showing greater response to abstract words in inferior prefrontal cortex. (3) The relationships between concrete words are governed primarily by conceptual similarity, while those of abstract words depend on association to a greater degree. This theory, based primarily on interference and priming effects in aphasic patients, is the most recent to emerge and the least well understood. I present analyses indicating that patterns of lexical co‐occurrence may be important in understanding these effects.     

In defense of abstractionist theories of repetition priming and word identification

There is a great deal of interest in characterizing the representations and processes that support visual word priming and written word identification more generally. On one view, these phenomena are supported by abstract orthographic representations that map together visually dissimilar exemplars of letters and words (e.g., the lettersA/a map onto a common abstract letter codea*). On a second view, orthographic codes consist in a collection of episodic representations of words that interact in such a way that it sometimes looksas if there are abstract codes. Tenpenny (1995) contrasted these general approaches and concluded by endorsing the episodic account, arguing that no evidence demands that we posit abstract orthographic representations. This review reconsiders the evidence and argues that a variety of priming and nonpriming research strongly supports the conclusion that abstract orthographic codes exist and support priming and word identification. On this account, episodic representations are represented separately from abstract orthographic knowledge and contribute minimally to these functions.     

Acalculia: An historical review of localization

This article reviews the brain localization of calculation disorders (acalculia) beginning with Gall's claim in the early 19th century for a "center" of calculation. A renewed interest in the subject arose around the time of Henschen during the first quarter of the 20th century. A summary of the cases of acalculia since Henschen leads to the conclusion that regardless of the functional modular nature of calculation ability, there is neither a localized region nor a specific hemisphere uniquely underlying the disorder.     

Modality switching costs emerge in concept creation as well as retrieval

Theories of embodied cognition hold that the conceptual system uses perceptual simulations for the purposes of representation. A strong prediction is that perceptual phenomena should emerge in conceptual processing, and, in support, previous research has shown that switching modalities from one trial to the next incurs a processing cost during conceptual tasks. However, to date, such research has been limited by its reliance on the retrieval of familiar concepts. We therefore examined concept creation by asking participants to interpret modality-specific compound phrases (i.e., conceptual combinations). Results show that modality switching costs emerge during the creation of new conceptual entities: People are slower to simulate a novel concept (e.g., auditory jingling onion) when their attention has already been engaged by a different modality in simulating a familiar concept (e.g., visual shiny penny). Furthermore, these costs cannot be accounted for by linguistic factors alone. Rather, our findings support the embodied view that concept creation, as well as retrieval, requires situated perceptual simulation.     

Perceptual processing affects conceptual processing

According to the Perceptual Symbols Theory of cognition ( Barsalou, 1999 ), modality-specific simulations underlie the representation of concepts. A strong prediction of this view is that perceptual processing affects conceptual processing. In this study, participants performed a perceptual detection task and a conceptual property-verification task in alternation. Responses on the property-verification task were slower for those trials that were preceded by a perceptual trial in a different modality than for those that were preceded by a perceptual trial in the same modality. This finding of a modality-switch effect across perceptual processing and conceptual processing supports the hypothesis that perceptual and conceptual representations are partially based on the same systems.     

Modality independence of order coding in working memory: Evidence from cross-modal order interference at recall

Working memory researchers do not agree on whether order in serial recall is encoded by dedicated modality-specific systems or by a more general modality-independent system. Although previous research supports the existence of autonomous modality-specific systems, it has been shown that serial recognition memory is prone to cross-modal order interference by concurrent tasks. The present study used a serial recall task, which was performed in a single-task condition and in a dual-task condition with an embedded memory task in the retention interval. The modality of the serial task was either verbal or visuospatial, and the embedded tasks were in the other modality and required either serial or item recall. Care was taken to avoid modality overlaps during presentation and recall. In Experiment 1, visuospatial but not verbal serial recall was more impaired when the embedded task was an order than when it was an item task. Using a more difficult verbal serial recall task, verbal serial recall was also more impaired by another order recall task in Experiment 2. These findings are consistent with the hypothesis of modality-independent order coding. The implications for views on short-term recall and the multicomponent view of working memory are discussed.     

The ineluctable modality of the audible: perceptual determinants of auditory verbal short-term memory

Classical cognitive accounts of verbal short-term memory (STM) invoke an abstract, phonological level of representation which, although it may be derived differently via different modalities, is itself amodal. Key evidence for this view is that serial recall of phonologically similar verbal items (e.g., the letter sounds b, c, g, and d) is worse than that of dissimilar items, regardless of modality of presentation. Here we show that the effect of such phonological similarity in STM can be fully accounted for by the joint action of articulatory similarity, leading to errors in speech planning processes, and acoustic similarity within auditorily presented lists, which modulates their perceptual organization. The results indicate that key evidence used to argue for the existence of abstract phonological representation can in fact be fully accounted for by reference to modality-specific perceptual and motor planning mechanisms.     

Cognitive and Neural Contributions to Understanding the Conceptual System

ABSTRACT— The conceptual system contains categorical knowledge about experience that supports the spectrum of cognitive processes. Cognitive science theories assume that categorical knowledge resides in a modular and amodal semantic memory, whereas neuroscience theories assume that categorical knowledge is grounded in the brain's modal systems for perception, action, and affect. Neuroscience has influenced theories of the conceptual system by stressing principles of neural processing in neural networks and by motivating grounded theories of cognition, which propose that simulations of experience represent knowledge. Cognitive science has influenced theories of the conceptual system by documenting conceptual phenomena and symbolic operations that must be grounded in the brain. Significant progress in understanding the conceptual system is most likely to occur if cognitive and neural approaches achieve successful integration.     

感知运动信息在概念表征中的作用

研究概念的表征问题对理解概念的本质非常重要,传统认知和具身认知视角下的概念表征理论争议的焦点在于感觉运动信息在表征中的作用。传统认知视角下的离身认知认为感知觉运动信息会转化成抽象的符号,概念表征不包含感知觉运动信息。概念表征的具身观点认为感知觉运动信息是概念表征的基础。对感知运动信息在概念表征中起作用这一命题已经达成共识。未来研究应该关注感知运动信息起作用的机制,以及抽象概念表征等问题,进一步完善发展概念表征理论。     

The concreteness effect: evidence for dual coding and context availability

The term concreteness effect refers to the observation that concrete nouns are processed faster and more accurately than abstract nouns in a variety of cognitive tasks. Two models have been proposed to explain the neuronal basis of the concreteness effect. The dual-coding theory attributes the advantage to the access of a right hemisphere image based system in addition to a verbal system by concrete words. The context availability theory argues that concrete words activate a broader contextual verbal support, which results in faster processing, but do not access a distinct image based system. We used event-related fMRI to detect the brain regions that subserve to the concreteness effect. We found greater activation in the lower right and left parietal lobes, in the left inferior frontal lobe and in the precuneus during encoding of concrete compared to abstract nouns. This makes a single exclusive theory unlikely and rather suggests a combination of both models. Superior encoding of concrete words in the present study may result from (1) greater verbal context resources reflected by the activation of left parietal and frontal associative areas, and (2) the additional activation of a non-verbal, perhaps spatial imagery-based system, in the right parietal lobe.     

The neurobiology of semantic memory

Semantic memory includes all acquired knowledge about the world and is the basis for nearly all human activity, yet its neurobiological foundation is only now becoming clear. Recent neuroimaging studies demonstrate two striking results: the participation of modality-specific sensory, motor, and emotion systems in language comprehension, and the existence of large brain regions that participate in comprehension tasks but are not modality-specific. These latter regions, which include the inferior parietal lobe and much of the temporal lobe, lie at convergences of multiple perceptual processing streams. These convergences enable increasingly abstract, supramodal representations of perceptual experience that support a variety of conceptual functions including object recognition, social cognition, language, and the remarkable human capacity to remember the past and imagine the future.     

An encoding-complex approach to numerical cognition in Chinese-English bilinguals.

We present a model of the cognitive architecture of basic numerical skills in adult Chinese-English bilinguals. The model is based on data reported by Campbell, Kanz, and Xue (1999) and combines Dehaene and Cohen's triple-code theory with Campbell and Clark's encoding-complex approach to modeling number processing. Participants were required to name, add or multiply Arabic or Mandarin numerals and to respond in English or Chinese. They also performed magnitude comparisons on pairs of Arabic or Mandarin numerals. The proposed model of their performance on this set of tasks assumes 1) that number processing is modular with respect to representational code (e.g., visual, visuo-spatial, verbal) rather than with respect to numerical function, 2) task-specific communication between representational codes is interactive rather than additive, and 3) memory for arithmetic facts is at least partially language-based and our Chinese-English bilinguals possessed both Chinese and English-language number-fact representations. We provide new analyses of the arithmetic data and a review of research on the role of language in simple arithmetic to substantiate our claims about linguistic codes for number-fact memory.     

Acalculia and Dyscalculia

Even though it is generally recognized that calculation ability represents a most important type of cognition, there is a significant paucity in the study of acalculia. In this paper the historical evolution of calculation abilities in humankind and the appearance of numerical concepts in child development are reviewed. Developmental calculation disturbances (developmental dyscalculia) are analyzed. It is proposed that calculation ability represents a multifactor skill, including verbal, spatial, memory, body knowledge, and executive function abilities. A general distinction between primary and secondary acalculias is presented, and different types of acquired calculation disturbances are analyzed. The association between acalculia and aphasia, apraxia and dementia is further considered, and special mention to the so-called Gerstmann syndrome is made. A model for the neuropsychological assessment of numerical abilities is proposed, and some general guidelines for the rehabilitation of calculation disturbances are presented.     

Modality-specific effects in inhibitory mechanisms: the interaction of peripheral and central components in working memory

Working memory is a complex system assumed to encompass both storage and processing components. At the same time, several authors have suggested the existence of separate peripheral modality-specific subsystems, in particular a verbal and a visuo-spatial mechanism. In the present research, we explored the hypothesis that modality-specific mechanisms may also operate in the central processing component. Inhibitory control has traditionally been considered as a typical function of the central executive, devoted to selecting, processing, and eliminating irrelevant information. We designed verbal and visuo-spatial tasks involving similar procedures and comparable levels of inhibitory control. Results indicate that accuracy profiles are similar in the two conditions; however, the patterns of data measuring inhibitory control, i.e., percentage of intrusion errors, are opposite in the case of verbal (intrusions decreased in more complex trials) and visuo-spatial (intrusions increased in more complex trials) material. These data are consistent with theoretical models of working memory suggesting that modality-specific effects emerge even in central processes and that both material-dependent and process-dependent features must be considered in working memory tasks.     

具身概念表征的研究及理论述评

具身认知强调认知在本质上是具身的, 身体在认知的实现中发挥着关键作用。传统的符号加工理论认为, 概念表征独立于主体的知觉运动系统并以抽象符号的形式储存于语言记忆中。概念表征的具身理论则认为, 概念表征与知觉运动系统具有共同的神经基础, 概念在本质上是主体经验客体时知觉与运动体验的神经记录, 而概念加工的基本形式则是身体经验的模拟与还原。关于该理论的实证研究主要集中于概念加工引发的知觉动作变化、身体动作对概念加工的影响、抽象概念加工的具身特征等领域。今后的研究应关注符号加工理论与具身理论的整合等。     

Data In Search Of A Principle: A Review Of Relational Frame Theory: A Post-Skinnerian Count Of Human Language And Cognition.

Responding to derived relations among stimuli and events is the subject of an accelerating research program that represents one of the major behavior analytic approaches to complex behavior. Relational Frame Theory: A Post‐Skinnerian Account of Human Language and Cognition (Hayes, Barnes‐Holmes, & Roche, 2001) offers a conceptual framework for this work and explores its implications for verbal behavior and a variety of other domains of complex human behavior. The authors dismiss Skinner's interpretation of verbal behavior as unproductive and conceptually flawed and suggest a new definition and a new paradigm for the investigation of verbal phenomena. I found the empirical phenomena important but the conceptual discussion incomplete. A new principle of behavior is promised, but critical features of this principle are not offered. In the absence of an explicit principle, the theory itself is difficult to evaluate. Counterexamples suggest a role for mediating behavior, perhaps covert, thus raising the question whether a new principle is needed at all. The performance of subjects in relational frame experiments may be a mosaic of elementary behavioral units, some of which are verbal. If so, verbal behavior underlies relational behavior; it is not defined by it. I defend Skinner's definition of verbal behavior and argue that an account of relational behavior must be integrated with Skinner's analysis; it will not replace it.     

Effects of concreteness and semantic relatedness on composite imagery ratings and cued recall

Day and Bellezza (1983) rejected a dual coding imagery explanation for the superior recall of concrete words because unrelated concrete pairs were rated lower in composite imagery but were still remembered better than related abstract pairs. We show that dual coding theory explains their results and our new findings using the same paradigm. In Experiment 1, 120 subjects rated imagery or relatedness for 108 pairs that varied in concreteness, pair relatedness, and associative strength. Incidental cued recall followed. Relatedness and strength affected imagery ratings, as did concreteness, and very low relatedness partly accounted for the low composite imagery ratings for unrelated concrete pairs. Concreteness and relatedness also affected recall, and suporior recall for unrelated concrete pairs occurred consistently under imagery but not under relatedness instructions. In Experiment 2, 40 subjects rated imagery value and recalled 24 pairs. Subsequent questioning indicated that composite images were retrieved better given stimuli from unrelated concrete than from related abstract pairs. These findings and Day and Bellezza’s original results are explained in terms of (1) imaginal and verbal associative processes, which jointly influence composite imagery ratings and recall, and (2) the critical role of stimulus concreteness during image retrieval and recall (i.e., the conceptual peg hypothesis).     

How semantic memory structure and intelligence contribute to creative thought: a network science approach

The associative theory of creativity states that creativity is associated with differences in the structure of semantic memory, whereas the executive theory of creativity emphasises the role of top-down control for creative thought. For a powerful test of these accounts, individual semantic memory structure was modelled with a novel method based on semantic relatedness judgements and different criteria for network filtering were compared. The executive account was supported by a correlation between creative ability and broad retrieval ability. The associative account was independently supported, when network filtering was based on a relatedness threshold, but not when it was based on a fixed edge number or on the analysis of weighted networks. In the former case, creative ability was associated with shorter average path lengths and higher clustering of the network, suggesting that the semantic networks of creative people show higher small-worldness.