Which body for embodied cognition? Affordance and language within actual and perceived reaching space

The mental representation of one's own body does not necessarily correspond to the physical body. For instance, a dissociation between perceived and actual reach-ability has been shown, that is, individuals perceive that they can reach objects that are out of grasp. We presented participants with 3D pictures of objects located at four different distances, namely near-reaching space, actual-reaching space, perceived-reaching space and non-reaching space. Immediately after they were presented with function, manipulation, observation or pointing verbs and were required to judge if the verb was compatible with the object. Participants were faster with function and manipulation verbs than with observation and pointing verbs. Strikingly, with both function and manipulation verbs participants were faster when objects were presented in actual than the perceived reaching space. These findings suggest that our knowledge of the world is implicitly built online through behaviour, and is not necessarily reflected in explicit estimates or conscious representations.     

A functional role for the motor system in language understanding: evidence from theta-burst transcranial magnetic stimulation

Does language comprehension depend, in part, on neural systems for action? In previous studies, motor areas of the brain were activated when people read or listened to action verbs, but it remains unclear whether such activation is functionally relevant for comprehension. In the experiments reported here, we used off-line theta-burst transcranial magnetic stimulation to investigate whether a causal relationship exists between activity in premotor cortex and action-language understanding. Right-handed participants completed a lexical decision task, in which they read verbs describing manual actions typically performed with the dominant hand (e.g., "to throw," "to write") and verbs describing nonmanual actions (e.g., "to earn," "to wander"). Responses to manual-action verbs (but not to nonmanual-action verbs) were faster after stimulation of the hand area in left premotor cortex than after stimulation of the hand area in right premotor cortex. These results suggest that premotor cortex has a functional role in action-language understanding.     

Effects of noun-verb homonymy on the neural correlates of naming concrete entities and actions

The neural correlates of naming concrete entities such as tools (with nouns) and naming actions (with verbs) are partially distinct: the former are linked to the left inferotemporal (IT) region, whereas the latter are linked to the left frontal opercular (FO) and left posterior middle temporal (MT) regions. This raises an intriguing question: How would such neural patterns be influenced by noun-verb homonymy, specifically, naming tasks in which the target words denote objects or actions (e.g., "comb")? To explore this, we conducted a PET study in which 10 normal participants named visually presented tools or actions. The factor of homonymy yielded interesting effects: For tools, non-homonymous nouns (e.g., "camera") activated left IT, whereas homonymous nouns (e.g., "comb") activated both left IT and left FO. For actions, non-homonymous (e.g., "juggle") and homonymous (e.g., "comb") verbs activated left FO, MT, and IT, but there was evidence that the FO and MT activations were less widespread for the homonymous verbs. We also found that retrieval of the same exact words (e.g., "comb" and "comb") produced differential activation in left MT-there was greater MT activation when the words were being used to name actions, than when they were being used to name tools. Our results suggest that noun-verb homonymy has an important influence on the patterns of neural activation associated with words denoting objects and actions, and that even when the phonological forms are identical, the patterns of neural activation are different according to the demands of the task.     

Actions and affordances in syntactic ambiguity resolution

In 2 experiments, eye movements were monitored as participants followed instructions containing temporary syntactic ambiguities (e.g., "Pour the egg in the bowl over the flour"). The authors varied the affordances of task-relevant objects with respect to the action required by the instruction (e.g., whether 1 or both eggs in the visual workspace were in liquid form, allowing them to be poured). The number of candidate objects that could afford the action was found to determine whether listeners initially misinterpreted the ambiguous phrase ("in the bowl") as specifying a location. The findings indicate that syntactic decisions are guided by the listener's situation-specific evaluation of how to achieve the behavioral goal of an utterance.     

Conflict in the kitchen: Contextual modulation of responsiveness to affordances

Recently, human behavior has been considered the product of continuous interactions between perception, cognition and action in which “affordances” (action possibilities the environment has to offer) play an important role. Converging evidence suggests that multiple action possibilities simultaneously compete for further processing, while external and internal factors (e.g., incoming sensory information, predictions) bias this competition. In the present study we used a stop-task to investigate whether context is able to modulate the strength of the responsiveness to affordances. We therefore placed participants in an actual kitchen and workshop during electroencephalographic recordings. A faster response to context congruent objects demonstrated that the direct surrounding is able to affect responsiveness to affordances. In addition, when responses needed to be withheld, context congruent objects evoked greater response conflict as indicated by an enhanced N2 Event Related Potential (ERP) component.     

Thematic role properties of subjects and objects

This paper tests two claims about the thematic roles Agent and Patient: first, that they can be decomposed into more primitive features, as laid out in Dowty's (1991) Proto-Roles Hypothesis; and second, that these properties can be inferred directly from the grammatical roles subject and object. In Experiments 1 and 2, participants rated the Proto-Roles properties of the subject and object of transitive sentences with real verbs. Subjects were rated as more Agent-like than objects, while objects were rated as more Patient-like than subjects. Experiment 3 used transitive sentences with nonsense content words (e.g. "The rom mecked the zarg"). Even in the absence of a known verb, the results were the same. Experiment 4 examined the corollary prediction that subjects and objects should not differ on grammatically irrelevant properties (e.g. "being liquid"); this prediction was confirmed. In Experiment 5, verbs that do not permit objects (e.g. "fall," "erupt") were placed in the transitive frame. Results were the same as in Experiments 1 through 3. Moreover, the semantics of the verb were altered by the frame, indicating that participants tried to fuse the semantics of the verb with those of the frame. Possible sources of these inferences are considered.     

Action matters: The role of action plans and object affordances in selection for action

In a series of three experiments requiring selection of real objects for action, we investigated whether characteristics of the planned action and/or the “affordances” of target and distractor objects affected interference caused by distractors. In all ofthe experiments, the target object was selectedon the basis of colour and was presented alone or with a distractor object. We examined the effect of type of response (button press, grasping, or pointing), object affordances (compatibility with the acting hand, affordances for grasping or pointing), and target/distractor positions (left or right) on distractor interference (reaction time differences between trials with and without distractors). Different patterns of distractor interference were associated with different motor responses. In the button-press conditions of each experiment, distractor interference was largely determined by perceptual salience (e.g., proximity to initial visual fixation). In contrast, in tasks requiring action upon the objects in the array, distractors with handles caused greater interference than those without handles, irrespective of whether the intended action was pointing or grasping. Additionally, handled distractors were relatively more salient when their affordances for grasping were strong (handle direction compatible with the acting hand) than when affordances were weak. These data suggest that attentional highlighting of specific target and distractor features is a function of intended actions.     

Artifacts and affordances

What are the affordances of artifacts? One view is that the affordances of artifacts, just as the affordances of natural objects, pertain to possible ways in which they can be manipulated (e.g., a computer keyboard affords grasping). Another view maintains that, given that artifacts are sociocultural objects, their affordances pertain primarily to their culturally-derived function (e.g., a computer keyboard affords typing). Whereas some have tried to provide a unifying notion of affordance to capture both aspects, here I argue that they should be kept separate. In this paper, I introduce a distinction between standard affordances, which concern the function of artifacts, and ad-hoc affordances, which refer to how artifacts are manipulated. I then argue for the neuropsychological plausibility of such a distinction, linking it to the dissociation between function knowledge and manipulation knowledge. Finally, I defend the equal status of these forms of knowledge and, hence, of standard and ad-hoc affordances, and I show that this has some implications for the debate on the role of motor processes in the conceptual knowledge of artifacts.

     

Grammatical category ambiguity in aphasia

This study asked whether aphasic adults show different noun/verb retrieval patterns based upon their clinical categorization as fluent or nonfluent. Participants selected either the noun or the verb meaning of target words, as presented in three contexts. The framework was that nouns (associated with temporal lobe function) are processed, stored, and retrieved separately from verbs (associated with frontal lobe function), implying separate status in the mental lexicon. Stimuli were homophonic homographs, words that are spelled and pronounced the same but which have different meanings (in this case, noun and verb meanings). Another contrast was the putative difference between systematic pairs (e.g., "kiss" and "farm"), in which noun and verb meanings are transparently related, and may be stored as a unit, and unsystematic pairs (e.g., "squash" and "sink"), in which noun and verb meanings are apparently unrelated, implying discrete storage. Results demonstrated significant interactions between fluent and nonfluent participants, suggesting that, as expected, fluent aphasic adults have more difficulty with nouns, nonfluent aphasic adults have more difficulty with verbs. There was no effect of systematicity. Contrary to expectations, verbs proved less vulnerable, rather than more vulnerable, to aphasic impairment.     

The eyes grasp,the hands see: Metric category knowledge transfers between vision and touch

Categorization of seen objects is often determined by the shapes of objects. However, shape is not exclusive to the visual modality: The haptic system also is expert at identifying shapes. Hence, an important question for understanding shape processing is whether humans store separate modality-dependent shape representations, or whether information is integrated into one multisensory representation. To answer this question, we created a metric space of computer-generated novel objects varying in shape. These objects were then printed using a 3-D printer, to generate tangible stimuli. In a categorization experiment, participants first explored the objects visually and haptically. We found that both modalities led to highly similar categorization behavior. Next, participants were trained either visually or haptically on shape categories within the metric space. As expected, visual training increased visual performance, and haptic training increased haptic performance. Importantly, however, we found that visual training also improved haptic performance, and vice versa. Two additional experiments showed that the location of the categorical boundary in the metric space also transferred across modalities, as did heightened discriminability of objects adjacent to the boundary. This observed transfer of metric category knowledge across modalities indicates that visual and haptic forms of shape information are integrated into a shared multisensory representation.     

The Role of Tentativeness in Perceiving Architecture and Art: A Far-from-Equilibrium Ecological Perspective

Two experiments examined performance in collision avoidance situations. In both experiments participants were asked to initiate a discrete maneuver to avoid a collision at the last possible moment. The affordances of the situations were varied as a function of vehicle dynamics and the functional consequences associated with responding too late or too early. The results were examined in the context of a 2-dimensional functional state space with dimensions associated with optical angle and optical expansion rate. The patterns of performance showed that the actions were consistent with decision rules that could be specified in terms of linear functions of the two optical variables. In most cases, performance at early stages of learning suggested that people were using an Expansion Rate Criterion. With practice, people would tune to a decision rule that was appropriate for the specific vehicle dynamics tested. The results are discussed in relation to the role of three factors in shaping ultimate performance: (a) tasks constraints (i.e., affordances), (b) information constraints (i.e., optical structure), and (c) experience (i.e., learning).     

Differential effects of cast shadows on perception and action

In two experiments we investigated the effects of cast shadows on different real-life tasks. In experiment 1, participants were required to make a speeded verbal identification of the target object (perceptual task), whereas in experiment 2 participants were required to reach for and grasp the target object (motor task). In both experiments real three-dimensional (3-D) objects were presented, one at a time, either with their own natural cast shadow (congruent condition) or with the cast shadow of a different object (incongruent condition). Shadows were cast either to the left or to the right of the object. We asked whether the features of the shadow (ie whether it is congruent or incongruent with the object, and whether it is cast to the left or to the right of the object) could influence perception and action differently. Results showed that cast shadows did not influence identification of real 3-D objects (experiment 1), but they affected movement kinematics, producing distractor-like interference, particularly on movement trajectory (experiment 2). These findings suggest a task-dependent influence of cast shadows on human performance. In the case of object-oriented actions, cast shadows may represent further affordances of the object, and as such compete for the control of the action.     

What is in a tool concept? Dissociating manipulation knowledge from function knowledge

Research on patients with apraxia, a deficit in skilled action, has shown that the ability to use objects may be differentially impaired relative to knowledge about object function. Here we show, using a modified neuropsychological test, that similar dissociations can be observed in response times in healthy adults. Participants were asked to decide which two of three presented objects shared the same manipulation or the same function; triads were presented in picture and word format, and responses were made manually (button press) or with a basic-level naming response (verbally). For manual responses (Experiment 1), participants were slower to make manipulation judgments for word stimuli than for picture stimuli, while there was no difference between word and picture stimuli for function judgments. For verbal-naming responses (Experiment 2), participants were again slower for manipulation judgments over word stimuli, as compared with picture stimuli; however, and in contrast to Experiment 1, function judgments over word stimuli were faster than function judgments over picture stimuli. These data support the hypotheses that knowledge of object function and knowledge of object manipulation correspond to dissociable types of object knowledge and that simulation over motor information is not necessary in order to retrieve knowledge of object function.     

Lateralization of object-shape information in semantic processing

An experiment was conducted to examine whether perceptual information, specifically the shape of objects, is activated during semantic processing. Subjects judged whether a target word was related to a prime word. Prime-target pairs that were not associated, but whose referents had similar shapes (e.g. LADDER-RAILROAD) yielded longer "no" responses than unassociated prime-target pairs, suggesting that shape information had been activated. A visual-field manipulation showed that, in right-handed subjects, this effect was localized in the left hemisphere. This finding is consistent with behavioral, brain imaging, and lesion data, which suggest that object shape at the category level is represented in the left hemisphere.     

Priming of reach and grasp actions by handled objects

Pictures of handled objects such as a beer mug or frying pan are shown to prime speeded reach and grasp actions that are compatible with the object. To determine whether the evocation of motor affordances implied by this result is driven merely by the physical orientation of the object's handle as opposed to higher-level properties of the object, including its function, prime objects were presented either in an upright orientation or rotated 90° from upright. Rotated objects successfully primed hand actions that fit the object's new orientation (e.g., a frying pan rotated 90° so that its handle pointed downward primed a vertically oriented power grasp), but only when the required grasp was commensurate with the object's proper function. This constraint suggests that rotated objects evoke motor representations only when they afford the potential to be readily positioned for functional action.     

The influence of athletic experience and kinematic information on skill-relevant affordance perception

Humans can perceive affordances both for themselves and for others, and affordance perception is a function of perceptual–motor experience involved in playing a sport. Two experiments investigated the enhanced affordance perception of athletes. In Experiment 1, basketball players and nonbasketball players provided perceptual reports for sports-relevant (maximum standing-reach and reach-with-jump heights) and non-sports-relevant (maximum sitting height) affordances for self and other. Basketball players were more accurate at perceiving maximum reach-with-jump for another person than were nonbasketball players, but were no better at perceiving maximum reach or sitting heights. Experiment 2 investigated the informational basis for this enhanced perceptual ability of basketball players by evaluating whether kinematics inform perceivers about action-scaled (e.g., force-production dependent), but not body-scaled (i.e., geometrically determined), affordances for others, and whether basketball experience enhances sensitivity to kinematic information. Only basketball players improved at perceiving an action-scaled affordance (maximum reach-with-jump), but not body-scaled affordances (maximum standing-reach and sit) with exposure to kinematic information, suggesting that action-scaled affordances may be specified by kinematic information to which athletes are already attuned by virtue of their sport experience.     

The influence of athletic experience and kinematic information on skill-relevant affordance perception

Humans can perceive affordances both for themselves and for others, and affordance perception is a function of perceptual-motor experience involved in playing a sport. Two experiments investigated the enhanced affordance perception of athletes. In Experiment 1, basketball players and nonbasketball players provided perceptual reports for sports-relevant (maximum standing-reach and reach-with-jump heights) and non-sports-relevant (maximum sitting height) affordances for self and other. Basketball players were more accurate at perceiving maximum reach-with-jump for another person than were nonbasketball players, but were no better at perceiving maximum reach or sitting heights. Experiment 2 investigated the informational basis for this enhanced perceptual ability of basketball players by evaluating whether kinematics inform perceivers about action-scaled (e.g., force-production dependent), but not body-scaled (i.e., geometrically determined), affordances for others, and whether basketball experience enhances sensitivity to kinematic information. Only basketball players improved at perceiving an action-scaled affordance (maximum reach-with-jump), but not body-scaled affordances (maximum standing-reach and sit) with exposure to kinematic information, suggesting that action-scaled affordances may be specified by kinematic information to which athletes are already attuned by virtue of their sport experience.     

The role of specificity in the lexical encoding of participants

In addition to information about phonology, morphology and syntax, lexical entries contain semantic information about participants (e.g., Agent). However, the traditional criteria for determining how much participant information is lexically encoded have proved unreliable. We have proposed two semantic criteria (obligatoriness and selectivity) that jointly identify the participants that are lexically encoded in verbs. We tested whether one of these criteria, semantic selectivity, makes psychologically real distinctions between participant information that is lexically encoded and participant information that is not. We examined how readers integrated syntactically optional WH-constituents in filler-gap sentences when the participant information conveyed by the WH-filler was specific to a restricted class of verbs (i.e., source locations) and when it was not (i.e., event locations). Our results provide support for the role of specificity in the lexical encoding of participant information of syntactically optional constituents.     

Adult Age Differences in Memory for Verbs and Nouns

Younger and older adults were asked to remember noun pairs (e.g., head – cap), verb pairs (e.g., bounce – throw), and verb-noun pairs (e.g., break – stick). For half of the pairs, participants used imagined objects and performed an action or series of related actions for each pair. For the other half of the pairs, participants read but did not perform the pairs. Free recall and cued recall tests revealed that age differences in memory for both performed and nonperformed items were larger for verbs than for nouns. The recall advantage of nouns over verbs was larger for older than for younger adults. Verbs are hypothesized to be more difficult for older adults to remember because they are more and less specific than nouns and because it is more difficult to integrate verbs with other words than to integrate nouns with other words.