Verifying different-modality properties for concepts produces switching costs

According to perceptual symbol systems, sensorimotor simulations underlie the representation of concepts. It follows that sensorimotor phenomena should arise in conceptual processing. Previous studies have shown that switching from one modality to another during perceptual processing incurs a processing cost. If perceptual simulation underlies conceptual processing, then verifying the properties of concepts should exhibit a switching cost as well. For example, verifying a property in the auditory modality (e.g., BLENDER-loud) should be slower after verifying a property in a different modality (e.g., CRANBERRIES-tart) than after verifying a property in the same modality (e.g., LEAVES-rustling). Only words were presented to subjects, and there were no instructions to use imagery. Nevertheless, switching modalities incurred a cost, analogous to the cost of switching modalities in perception. A second experiment showed that this effect was not due to associative priming between properties in the same modality. These results support the hypothesis that perceptual simulation underlies conceptual processing.     

Sensory load incurs conceptual processing costs

Theories of grounded cognition propose that modal simulations underlie cognitive representation of concepts [Barsalou, L. W. (1999). Perceptual symbol systems. Behavioral and Brain Sciences, 22(4), 577-660; Barsalou, L. W. (2008). Grounded cognition. Annual Review of Psychology, 59, 617-645]. Based on recent evidence of modality-specific resources in perception, we hypothesized that verifying properties of concepts encoded in different modalities are hindered more by perceptual short-term memory load to the same versus different sensory modality as that used to process the property. We manipulated load to visual and auditory modalities by having participants store one or three items in short-term memory during property verification. In the high (but not low) load condition, property verification took longer when the property (e.g., yellow) involved the same modality as that used by the memory load (e.g., pictures). Interestingly, similar interference effects were obtained on the conceptual verification and on the memory task. These findings provide direct support for the view that conceptual processing relies on simulation in modality-specific systems.     

Modality switching costs emerge in concept creation as well as retrieval

Theories of embodied cognition hold that the conceptual system uses perceptual simulations for the purposes of representation. A strong prediction is that perceptual phenomena should emerge in conceptual processing, and, in support, previous research has shown that switching modalities from one trial to the next incurs a processing cost during conceptual tasks. However, to date, such research has been limited by its reliance on the retrieval of familiar concepts. We therefore examined concept creation by asking participants to interpret modality-specific compound phrases (i.e., conceptual combinations). Results show that modality switching costs emerge during the creation of new conceptual entities: People are slower to simulate a novel concept (e.g., auditory jingling onion) when their attention has already been engaged by a different modality in simulating a familiar concept (e.g., visual shiny penny). Furthermore, these costs cannot be accounted for by linguistic factors alone. Rather, our findings support the embodied view that concept creation, as well as retrieval, requires situated perceptual simulation.     

Switching between sensory and affective systems incurs processing costs

Recent models of the conceptual system hold that concepts are grounded in simulations of actual experiences with instances of those concepts in sensory-motor systems (e.g., Barsalou, 1999, 2003; Solomon & Barsalou, 2001). Studies supportive of such a viewhave shown that verifying a property of a concept in one modality, and then switching to verify a property of a different concept in a different modality generates temporal processing costs similar to the cost of switching modalities in perception. In addition to non-emotional concepts, the present experiment investigated switching costs in verifying properties of positive and negative (emotional) concepts. Properties of emotional concepts were taken from vision, audition, and the affective system. Parallel to switching costs in neutral concepts, the study showed that for positive and negative concepts, verifying properties from different modalities produced processing costs such that reaction times were longer and error rates were higher. Importantly, this effect was observed when switching from the affective system to sensory modalities, and vice-versa. These results support the embodied cognition view of emotion in humans.     

Perceptual symbol systems

Prior to the twentieth century, theories of knowledge were inherently perceptual. Since then, developments in logic, statistics, and programming languages have inspired amodal theories that rest on principles fundamentally different from those underlying perception. In addition, perceptual approaches have become widely viewed as untenable because they are assumed to implement recording systems, not conceptual systems. A perceptual theory of knowledge is developed here in the context of current cognitive science and neuroscience. During perceptual experience, association areas in the brain capture bottom-up patterns of activation in sensory-motor areas. Later, in a top-down manner, association areas partially reactivate sensory-motor areas to implement perceptual symbols. The storage and reactivation of perceptual symbols operates at the level of perceptual components--not at the level of holistic perceptual experiences. Through the use of selective attention, schematic representations of perceptual components are extracted from experience and stored in memory (e.g., individual memories of green, purr, hot). As memories of the same component become organized around a common frame, they implement a simulator that produces limitless simulations of the component (e.g., simulations of purr). Not only do such simulators develop for aspects of sensory experience, they also develop for aspects of proprioception (e.g., lift, run) and introspection (e.g., compare, memory, happy, hungry). Once established, these simulators implement a basic conceptual system that represents types, supports categorization, and produces categorical inferences. These simulators further support productivity, propositions, and abstract concepts, thereby implementing a fully functional conceptual system. Productivity results from integrating simulators combinatorially and recursively to produce complex simulations. Propositions result from binding simulators to perceived individuals to represent type-token relations. Abstract concepts are grounded in complex simulations of combined physical and introspective events. Thus, a perceptual theory of knowledge can implement a fully functional conceptual system while avoiding problems associated with amodal symbol systems. Implications for cognition, neuroscience, evolution, development, and artificial intelligence are explored.     

Modality exclusivity norms for 400 nouns: The relationship between perceptual experience and surface word form

We present modality exclusivity norms for 400 randomly selected noun concepts, for which participants provided perceptual strength ratings across five sensory modalities (i.e., hearing, taste, touch, smell, and vision). A comparison with previous norms showed that noun concepts are more multimodal than adjective concepts, as nouns tend to subsume multiple adjectival property concepts (e.g., perceptual experience of the concept baby involves auditory, haptic, olfactory, and visual properties, and hence leads to multimodal perceptual strength). To show the value of these norms, we then used them to test a prediction of the sound symbolism hypothesis: Analysis revealed a systematic relationship between strength of perceptual experience in the referent concept and surface word form, such that distinctive perceptual experience tends to attract distinctive lexical labels. In other words, modality-specific norms of perceptual strength are useful for exploring not just the nature of grounded concepts, but also the nature of form–meaning relationships. These norms will be of benefit to those interested in the representational nature of concepts, the roles of perceptual information in word processing and in grounded cognition more generally, and the relationship between form and meaning in language development and evolution.     

An asymmetric effect of relational integration on recognition memory

Two experiments assessed whether conceptual relations (e.g., contains: COOKIE JAR) facilitate the retrieval of concepts (e.g., COOKIE and JAR) from long-term memory. The CARIN model of nominal combination asserts that conceptual relations are represented with and selected from the modifier noun (COOKIE). Thus, the model predicts that relational integration will facilitate memory for modifiers but not for head nouns (JAR). In Experiment 1, concepts were studied in pairs and were then tested individually. Recognition accuracy was higher for modifiers than for head nouns. In Experiment 2, the studied concepts (e.g., COOKIE JAR) were tested in the context of a new pair that instantiated either the same relation (e.g., COOKIE plate) or a different relation (e.g., COOKIE crumb). Recognition was again better for modifiers than for head nouns, but only when the same conceptual relation was instantiated at both study and test. Thus, results indicate that conceptual relations (a) facilitate recognition memory, and (b) are associated more strongly with the modifier than with the head noun.     

Verifying visual properties in sentence verification facilitates picture recognition memory

According to the perceptual symbols theory (Barsalou, 1999), sensorimotor simulations underlie the representation of concepts. We investigated whether recognition memory for pictures of concepts was facilitated by earlier representation of visual properties of those concepts. During study, concept names (e.g., apple) were presented in a property verification task with a visual property (e.g., shiny) or with a nonvisual property (e.g., tart). Delayed picture recognition memory was better if the concept name had been presented with a visual property than if it had been presented with a nonvisual property. These results indicate that modality-specific simulations are used for concept representation.     

Are Automatic Conceptual Cores the Gold Standard of Semantic Processing? The Context‐Dependence of Spatial Meaning in Grounded Congruency Effects

According to grounded cognition, words whose semantics contain sensory‐motor features activate sensory‐motor simulations, which, in turn, interact with spatial responses to produce grounded congruency effects (e.g., processing the spatial feature of up for sky should be faster for up vs. down responses). Growing evidence shows these congruency effects do not always occur, suggesting instead that the grounded features in a word's meaning do not become active automatically across contexts. Researchers sometimes use this as evidence that concepts are not grounded, further concluding that grounded information is peripheral to the amodal cores of concepts. We first review broad evidence that words do not have conceptual cores, and that even the most salient features in a word's meaning are not activated automatically. Then, in three experiments, we provide further evidence that grounded congruency effects rely dynamically on context, with the central grounded features in a concept becoming active only when the current context makes them salient. Even when grounded features are central to a word's meaning, their activation depends on task conditions.     

A sharp image or a sharp knife: norms for the modality-exclusivity of 774 concept-property items

According to recent embodied cognition theories, mental concepts are represented by modality-specific sensory-motor systems. Much of the evidence for modality-specificity in conceptual processing comes from the property-verification task. When applying this and other tasks, it is important to select items based on their modality-exclusivity. We collected modality ratings for a set of 387 properties, each of which was paired with two different concepts, yielding a total of 774 concept-property items. For each item, participants rated the degree to which the property could be experienced through five perceptual modalities (vision, audition, touch, smell, and taste). Based on these ratings, we computed a measure of modality exclusivity, the degree to which a property is perceived exclusively through one sensory modality. In this paper, we briefly sketch the theoretical background of conceptual knowledge, discuss the use of the property-verification task in cognitive research, provide our norms and statistics, and validate the norms in a memory experiment. We conclude that our norms are important for researchers studying modality-specific effects in conceptual processing.     

感知运动信息在概念表征中的作用

研究概念的表征问题对理解概念的本质非常重要,传统认知和具身认知视角下的概念表征理论争议的焦点在于感觉运动信息在表征中的作用。传统认知视角下的离身认知认为感知觉运动信息会转化成抽象的符号,概念表征不包含感知觉运动信息。概念表征的具身观点认为感知觉运动信息是概念表征的基础。对感知运动信息在概念表征中起作用这一命题已经达成共识。未来研究应该关注感知运动信息起作用的机制,以及抽象概念表征等问题,进一步完善发展概念表征理论。     

Language Processing Differences Between Blind and Sighted Individuals and the Abstract Versus Concrete Concept Difference

In the property listing task (PLT), participants are asked to list properties for a concept (e.g., for the concept dog, “barks,” and “is a pet” may be produced). In conceptual property norming (CPNs) studies, participants are asked to list properties for large sets of concepts. Here, we use a mathematical model of the property listing process to explore two longstanding issues: characterizing the difference between concrete and abstract concepts, and characterizing semantic knowledge in the blind versus sighted population. When we apply our mathematical model to a large CPN reporting properties listed by sighted and blind participants, the model uncovers significant differences between concrete and abstract concepts. Though we also find that blind individuals show many of the same processing differences between abstract and concrete concepts found in sighted individuals, our model shows that those differences are noticeably less pronounced than in sighted individuals. We discuss our results vis-a-vis theories attempting to characterize abstract concepts.     

Perceptual processing affects conceptual processing

According to the Perceptual Symbols Theory of cognition ( Barsalou, 1999 ), modality-specific simulations underlie the representation of concepts. A strong prediction of this view is that perceptual processing affects conceptual processing. In this study, participants performed a perceptual detection task and a conceptual property-verification task in alternation. Responses on the property-verification task were slower for those trials that were preceded by a perceptual trial in a different modality than for those that were preceded by a perceptual trial in the same modality. This finding of a modality-switch effect across perceptual processing and conceptual processing supports the hypothesis that perceptual and conceptual representations are partially based on the same systems.     

Verifying properties of concepts spontaneously requires sharing resources with same-modality percept

In the present experiments, participants had to verify properties of concepts but, depending on the trial condition, concept-property pairs were presented via headphones or on the screen. The results showed that participants took longer and were less accurate at verifying conceptual properties when the channel used to present the CONCEPT-property pair and the type of property matched in sensory modality (e.g., LEMON-yellow on screen; BLENDER-loud in headphones) compared to when properties and channel did not match (e.g., LEMON-yellow in headphones; BLENDER-loud on screen). Such interference is consistent with theories of embodied cognition holding that knowledge is grounded in modality-specific systems (Barsalou in Behav Brain Sci 22:577–660, 1999). When the resources of one modality are burdened during the task, processing costs are incurred in a conceptual task (Vermeulen et al. in Cognition 109:287–294, 2008).     

Content differences for abstract and concrete concepts

Concept properties are an integral part of theories of conceptual representation and processing. To date, little is known about conceptual properties of abstract concepts, such as idea. This experiment systematically compared the content of 18 abstract and 18 concrete concepts, using a feature generation task. Thirty-one participants listed characteristics of the concepts (i.e., item properties) or their relevant context (i.e., context properties). Abstract concepts had significantly fewer intrinsic item properties and more properties expressing subjective experiences than concrete concepts. Situation components generated for abstract and concrete concepts differed in kind, but not in number. Abstract concepts were predominantly related to social aspects of situations. Properties were significantly less specific for abstract than for concrete concepts. Thus, abstractness emerged as a function of several, both qualitative and quantitative, factors.     

Object manipulability affects children’s and adults’ conceptual processing

Research on kinds of concepts indicates that children use perceptual and functional information differently to form natural and artifact concepts. Beyond object domain, object manipulability appears to be a decisive factor in adult conceptual processing. Thus, the effect of object manipulability on conceptual processing was tested in 5- and 7-year-olds and adults using a picture matching task. Reaction times for identifying conceptual relations on the basis of perceptual similarity (e.g., jacket-coat) and contextual/functional information (e.g., jacket-hanger) were analyzed according to object manipulability and domain. Both children and adults were faster to identify contextual/functional relations for manipulable than for nonmanipulable objects. Conversely, they were faster to identify perceptual similarity relations for nonmanipulable than for manipulable objects, particularly for natural concepts. Results reveal an early distinction between concepts of manipulable and nonmanipulable objects. Implications for further research on concept formation and for embodied views of concepts are discussed.     

Concepts are not represented by conscious imagery

According to theories of grounded cognition, conceptual representation and perception share processing mechanisms. We investigated whether this overlap is due to conscious perceptual imagery. Participants filled out questionnaires to assess the vividness of their imagery (Questionnaire on Mental Imagery) and the extent to which their imagery was object oriented and spatially oriented (Object-Spatial Imagery Questionnaire), and they performed a mental rotation task. One week later, they performed a verbal property verification task. In this task, involvement of modality-specific systems is evidenced by the modality-switch effect, the finding that performance on a target trial (e.g., apple—green) is better after a same-modality trial (e.g., diamond—sparkle) than after a different-modality trial (e.g., airplane—noisy). Results showed a modality-switch effect, but there was no systematic relation between imagery scores and modality switch. We conclude that conscious mental imagery is not fundamental to conceptual representation.     

The Impact of Analogies on Creative Concept Generation: Lessons From an In Vivo Study in Engineering Design

Research on innovation often highlights analogies from sources outside the current problem domain as a major source of novel concepts; however, the mechanisms underlying this relationship are not well understood. We analyzed the temporal interplay between far analogy use and creative concept generation in a professional design team's brainstorming conversations, investigating the hypothesis that far analogies lead directly to very novel concepts via large steps in conceptual spaces (jumps). Surprisingly, we found that concepts were more similar to their preceding concepts after far analogy use compared to baseline situations (i.e., without far analogy use). Yet far analogies increased the team's concept generation rate compared to baseline conditions. Overall, these results challenge the view that far analogies primarily lead to novel concepts via jumps in conceptual spaces and suggest alternative pathways from far analogies to novel concepts (e.g., iterative, deep exploration within a functional space).     

What is a Leather Iron or a Bird Phone? Using Conceptual Combinations to Generate and Understand New Product Concepts

This article introduces the framework of conceptual combinations, which underlies the creative ability to combine existing concepts to create new ones. Using this framework, two creative processes are identified, namely, (a) property mapping (PM), which entails combining concepts by transferring a property from one concept to another (e.g., shape in the case of notebook computers); and (b) relation linking (RL), which entails linking the two combining concepts by a thematic relation (e.g., the “locative” relation in desktop computers). The effect of these processes on the comprehension of new product concepts is investigated in two experimental studies. In Study 1 it is shown that novel products created by RL are easier to interpret than the ones created by PM. In Study 2 it is found that new products combining concepts from different super‐ordinate categories are more likely interpreted by RL, and are easier to comprehend than the ones from the same super‐ordinate category, which use PM. The theoretical and managerial implications of using conceptual combinations in the context of new product ideation are discussed.