Word meaning and the control of eye fixation: semantic competitor effects and the visual world paradigm

When participants are presented simultaneously with spoken language and a visual display depicting objects to which that language refers, participants spontaneously fixate the visual referents of the words being heard [Cooper, R. M. (1974). The control of eye fixation by the meaning of spoken language: A new methodology for the real-time investigation of speech perception, memory, and language processing. Cognitive Psychology, 6(1), 84-107; Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., & Sedivy, J. C. (1995). Integration of visual and linguistic information in spoken language comprehension. Science, 268(5217), 1632-1634]. We demonstrate here that such spontaneous fixation can be driven by partial semantic overlap between a word and a visual object. Participants heard the word ‘piano’ when (a) a piano was depicted amongst unrelated distractors; (b) a trumpet was depicted amongst those same distractors; and (c), both the piano and trumpet were depicted. The probability of fixating the piano and the trumpet in the first two conditions rose as the word ‘piano’ unfolded. In the final condition, only fixations to the piano rose, although the trumpet was fixated more than the distractors. We conclude that eye movements are driven by the degree of match, along various dimensions that go beyond simple visual form, between a word and the mental representations of objects in the concurrent visual field.     

Binocular vision enhances a rapidly evolving affordance priming effect: Behavioural and TMS evidence

Extensive research has suggested that simply viewing an object can automatically prime compatible actions for object manipulation, known as affordances. Here we explored the generation of covert motor plans afforded by real objects with precision (‘pinchable’) or whole-hand/power (‘graspable’) grip significance under different types of vision. In Experiment 1, participants viewed real object primes either monocularly or binocularly and responded to orthogonal auditory stimuli by making precision or power grips. Pinchable primes facilitated congruent precision grip responses relative to incongruent power grips, and vice versa for graspable primes, but only in the binocular vision condition. To examine the temporal evolution of the binocular affordance effect, participants in Experiment 2 always viewed the objects binocularly but made no responses, instead receiving a transcranial magnetic stimulation pulse over their primary motor cortex at three different times (150, 300, 450 ms) after prime onset. Motor evoked potentials (MEPs) recorded from a pinching muscle were selectively increased when subjects were primed with a pinchable object, whereas MEPs from a muscle associated with power grips were increased when viewing graspable stimuli. This interaction was obtained both 300 and 450 ms (but not 150 ms) after the visual onset of the prime, characterising for the first time the rapid development of binocular grip-specific affordances predicted by functional accounts of the affordance effect.     

Audiovisual semantic interference and attention: Evidence from the attentional blink paradigm

In the present study we investigate the role of attention in audiovisual semantic interference, by using an attentional blink paradigm. Participants were asked to make an unspeeded response to the identity of a visual target letter. This target letter was preceded at various SOAs by a synchronized audiovisual letter-pair, which was either congruent (e.g. hearing an “F” and viewing an “F”) or incongruent (e.g. hearing an “F” and viewing a “Z”). In Experiment 1, participants were asked to match the members of the audiovisual letter-pair. In Experiment 2, participants were asked to ignore the synchronized audiovisual letter-pairs altogether and only report the visual target. In Experiment 3, participants were asked to identify only one of the audiovisual letters (identify the auditory letter, and ignore the synchronized visual letter, or vice versa). An attentional blink was found in all three experiments indicating that the audiovisual letter-pairs were processed. However, a congruency effect on subsequent target detection was observed in Experiments 1 and 3, but not in Experiment 2. The results indicate that attention to the semantic contents of at least one modality is necessary to establish audiovisual semantic interference.     

Object-based auditory facilitation of visual search for pictures and words with frequent and rare targets

Auditory and visual processes demonstrably enhance each other based on spatial and temporal coincidence. Our recent results on visual search have shown that auditory signals also enhance visual salience of specific objects based on multimodal experience. For example, we tend to see an object (e.g., a cat) and simultaneously hear its characteristic sound (e.g., “meow”), to name an object when we see it, and to vocalize a word when we read it, but we do not tend to see a word (e.g., cat) and simultaneously hear the characteristic sound (e.g., “meow”) of the named object. If auditory–visual enhancements occur based on this pattern of experiential associations, playing a characteristic sound (e.g., “meow”) should facilitate visual search for the corresponding object (e.g., an image of a cat), hearing a name should facilitate visual search for both the corresponding object and corresponding word, but playing a characteristic sound should not facilitate visual search for the name of the corresponding object. Our present and prior results together confirmed these experiential association predictions. We also recently showed that the underlying object-based auditory–visual interactions occur rapidly (within 220 ms) and guide initial saccades towards target objects. If object-based auditory–visual enhancements are automatic and persistent, an interesting application would be to use characteristic sounds to facilitate visual search when targets are rare, such as during baggage screening. Our participants searched for a gun among other objects when a gun was presented on only 10% of the trials. The search time was speeded when a gun sound was played on every trial (primarily on gun-absent trials); importantly, playing gun sounds facilitated both gun-present and gun-absent responses, suggesting that object-based auditory–visual enhancements persistently increase the detectability of guns rather than simply biasing gun-present responses. Thus, object-based auditory–visual interactions that derive from experiential associations rapidly and persistently increase visual salience of corresponding objects.     

Mighty metaphors: behavioral and ERP evidence that power shifts attention on a vertical dimension

Thinking about the abstract concept power may automatically activate the spatial up-down image schema (powerful up; powerless down) and consequently direct spatial attention to the image schema-congruent location. Participants indicated whether a word represented a powerful or powerless person (e.g. ‘king’ or ‘servant’). Following each decision, they identified a target at the top or bottom of the visual field. In Experiment 1 participants identified the target faster when their spatial position was congruent with the perceived power of the preceding word than when it was incongruent. In Experiment 2 ERPs showed a higher N1 amplitude for congruent spatial positions. These results support the view that attention is driven to the image schema congruent location of a power word. Thus, power is partially understood in terms of vertical space, which demonstrates that abstract concepts are grounded in sensory-motor processing.     

Rapid word learning in 13- and 17-month-olds in a naturalistic two-word procedure: looking versus reaching measures

This study investigated infants’ rapid learning of two novel words using a preferential looking measure compared with a preferential reaching measure. In Experiment 1, 21 13-month-olds and 20 17-month-olds were given 12 novel label exposures (6 per trial) for each of two novel objects. Next, in the label comprehension tests, infants were shown both objects and were asked, “Where’s the [label]?” (looking preference) and then told, “Put the [label] in the basket” (reaching preference). Only the 13-month-olds showed rapid word learning on the looking measure; neither age group showed rapid word learning on the reaching measure. In Experiment 2, the procedure was repeated 24 h later with 10 participants per age group from Experiment 1. After a further 12 labels per object, both age groups now showed robust evidence of rapid word learning, but again only on the looking measure. This is the earliest looking-based evidence of rapid word learning in infants in a well-controlled (i.e., two-word) procedure; our failure to replicate previous reports of rapid word learning in 13-month-olds with a preferential reaching measure may be due to our use of more rigorous controls for object preferences. The superior performance of the younger infants on the looking measure in Experiment 1 was not straightforwardly predicted by existing theoretical accounts of word learning.     

Testing the embodied account of object naming: A concurrent motor task affects naming artifacts and animals

Embodied theories of object representation propose that the same neural networks are involved in encoding and retrieving object knowledge. In the present study, we investigated whether motor programs play a causal role in the retrieval of object names. Participants performed an object-naming task while squeezing a sponge with either their right or left hand. The objects were artifacts (e.g. hammer) or animals (e.g. giraffe) and were presented in an orientation that favored a grasp or not. We hypothesized that, if activation of motor programs is necessary to retrieve object knowledge, then concurrent motor activity would interfere with naming manipulable artifacts but not non-manipulable animals. In Experiment 1, we observed naming interference for all objects oriented towards the occupied hand. In Experiment 2, we presented the objects in more ‘canonical orientations’. Participants named all objects more quickly when they were oriented towards the occupied hand. Together, these interference/facilitation effects suggest that concurrent motor activity affects naming for both categories. These results also suggest that picture-plane orientation interacts with an attentional bias that is elicited by the objects and their relationship to the occupied hand. These results may be more parsimoniously accounted for by a domain-general attentional effect, constraining the embodied theory of object representations. We suggest that researchers should scrutinize attentional accounts of other embodied cognitive effects.     

The developing role of prosody in novel word interpretation

This study examined whether children use prosodic correlates to word meaning when interpreting novel words. For example, do children infer that a word spoken in a deep, slow, loud voice refers to something larger than a word spoken in a high, fast, quiet voice? Participants were 4- and 5-year-olds who viewed picture pairs that varied along a single dimension (e.g., big vs. small flower) and heard a recorded voice asking them, for example, “Can you get the blicket one?” spoken with either meaningful or neutral prosody. The 4-year-olds failed to map prosodic cues to their corresponding meaning, whereas the 5-year-olds succeeded (Experiment 1). However, 4-year-olds successfully mapped prosodic cues to word meaning following a training phase that reinforced children’s attention to prosodic information (Experiment 2). These studies constitute the first empirical demonstration that young children are able to use prosody-to-meaning correlates as a cue to novel word interpretation.     

What is in a tool concept? Dissociating manipulation knowledge from function knowledge

Research on patients with apraxia, a deficit in skilled action, has shown that the ability to use objects may be differentially impaired relative to knowledge about object function. Here we show, using a modified neuropsychological test, that similar dissociations can be observed in response times in healthy adults. Participants were asked to decide which two of three presented objects shared the same manipulation or the same function; triads were presented in picture and word format, and responses were made manually (button press) or with a basic-level naming response (verbally). For manual responses (Experiment 1), participants were slower to make manipulation judgments for word stimuli than for picture stimuli, while there was no difference between word and picture stimuli for function judgments. For verbal-naming responses (Experiment 2), participants were again slower for manipulation judgments over word stimuli, as compared with picture stimuli; however, and in contrast to Experiment 1, function judgments over word stimuli were faster than function judgments over picture stimuli. These data support the hypotheses that knowledge of object function and knowledge of object manipulation correspond to dissociable types of object knowledge and that simulation over motor information is not necessary in order to retrieve knowledge of object function.     

Resolving competition when naming an object in a multiple-object display

Naming an object in the context of other objects requires the selection and processing of the target object at different levels, while the processing of competing representations activated by context objects has to be constrained. At what stage are these competing representations attenuated? To address this question, we presented pairs of target and context objects that were either similar in visual shape (e.g., umbrella–palm tree) or dissimilar in visual shape (e.g., umbrella–ladder), so that the context object would attract various amounts of attention. The activation of the context object at different levels of processing was assessed by means of auditory distractors (semantically related, or phonologically related, or unrelated to the context object). Semantic and phonological distractor effects were observed for shape-related object pairs, but not for unrelated object pairs. This finding suggests that context objects do not activate their associated lexical representations to any substantial amount, unless they capture attention. In that case, they undergo full lexical processing up to a phonological level. Implications for models of word production are discussed.     

Mere acceptance produces apparent attitude in the Implicit Association Test

Two experiments examined mere acceptance effects in the Implicit Association Test (IAT). They tested whether accepting a stimulus as conforming to a rule produces responding consistent with positive attitude in the IAT. In Experiment 1, accepted stimuli were more easily categorized with pleasant personality characteristics than rejected stimuli; they were preferred according to the logic of the IAT. Accepted word stimuli were also responded to faster overall, suggesting that it was easier to make the accept than the reject response. In Experiment 2, numerical stimuli that conformed to a rule showed the same IAT preference effect over non-confirming stimuli, even when the rule conforming stimuli were more difficult to categorize. Three sources of this apparent preference for rule-conforming stimuli are considered: (1) the semantic relatedness of the concepts “accept” and “pleasant” on the one hand, and “reject” and “unpleasant” on the other; (2) that rejected non-category members are more salient (‘pop-out’) and thus are more easily categorized with the more salient unpleasant personality characteristics; or (3) that accepting rule-conforming stimuli is experienced as a pleasant event. Regardless of the mechanism underlying the mere acceptance effect, the IAT can produce apparent preferences for stimuli towards which participants have no positive attitude.     

汉语口语句子产生的语法编码计划单元

语法编码的计划单元作为口语句子产生领域的一个重要命题,近年来备受心理语言学者的关注。本研究利用词图干扰范式下的语义效应,通过两个行为实验探讨了汉语口语句子产生中的语法编码计划单元。实验中要求被试用介词短语做主语的句子或者并列名词短语做主语的句子来命名两个竖直排列的图片,并视觉呈现干扰词。结果表明,对于两个句式而言,在第二个名词上都没有潜伏期的语义干扰效应,只有错误率上的语义干扰效应,而且这个结果并没有随着SOA(目标图片和干扰词开始呈现的时间间隔)的改变(0、150或300ms)而改变。当干扰词与图片同时呈现时(SOA=0)在并列名词短语做主语的句子中,第二个名词上出现了潜伏期的语义促进效应。这些结果说明,语法编码的说法较为宽泛,应具体描述为:词汇选择是严格递进的,而概念激活的计划单元可能是功能短语。     

Spatial distance effects on incremental semantic interpretation of abstract sentences: Evidence from eye tracking

A large body of evidence has shown that visual context information can rapidly modulate language comprehension for concrete sentences and when it is mediated by a referential or a lexical-semantic link. What has not yet been examined is whether visual context can also modulate comprehension of abstract sentences incrementally when it is neither referenced by, nor lexically associated with, the sentence. Three eye-tracking reading experiments examined the effects of spatial distance between words (Experiment 1) and objects (Experiment 2 and 3) on participants’ reading times for sentences that convey similarity or difference between two abstract nouns (e.g., ‘Peace and war are certainly different...’). Before reading the sentence, participants inspected a visual context with two playing cards that moved either far apart or close together. In Experiment 1, the cards turned and showed the first two nouns of the sentence (e.g., ‘peace’, ‘war’). In Experiments 2 and 3, they turned but remained blank. Participants’ reading times at the adjective (Experiment 1: first-pass reading time; Experiment 2: total times) and at the second noun phrase (Experiment 3: first-pass times) were faster for sentences that expressed similarity when the preceding words/objects were close together (vs. far apart) and for sentences that expressed dissimilarity when the preceding words/objects were far apart (vs. close together). Thus, spatial distance between words or entirely unrelated objects can rapidly and incrementally modulate the semantic interpretation of abstract sentences.     

Effects of semantic priming on visual encoding of pictures

We investigated the effects of semantic priming on initial encoding of briefly presented pictures of objects and scenes. Pictures in four experiments were presented for varying durations and were followed immediately by a mask. In Experiments 1 and 2, pictures of simple objects were either preceded or not preceded by the object's category name (e.g., dog). In Experiment 1 we measured immediate object identification; in Experiment 2 we measured delayed old/new recognition in which targets and distractors were from the same categories. In Experiment 3 naturalistic scenes were either preceded or not preceded by the scene's category name (e.g., supermarket). We measured delayed recognition in which targets and distractors were described by the same category names. In Experiments 1-3, performance was better for primed than for unprimed pictures. Experiment 4 was similar to Experiment 2 in that we measured delayed recognition for simple objects. As in Experiments 1-3, a prime that preceded the object improved subsequent memory performance for the object. However, a prime that followed the object did not affect subsequent performance. Together, these results imply that priming leads to more efficient information acquisition. We offer a picture-processing model that accounts for these results. The model's central assumption is that knowledge of a picture's category (gist) increases the rate at which visual information is acquired from the picture.     

Will Julia Roberts harm Nicole Kidman? Semantic priming effects during face naming

Three experiments are reported, which examine whether face naming is vulnerable to semantic competition in a similar way to object naming. Previous experiments with object naming have shown that a related prime picture presented 3 trials before a target picture results in an increase in error rate and naming latencies when compared to unrelated prime conditions. The experiments here use the same paradigm, with errors as the main dependent variable. In Experiment 1, the prime and target faces were from the same occupational category (e.g., politicians, actors), and in Experiment 2, the primes and target faces were also associated to each other. In Experiment 3, the prime was presented as a name to be read aloud. Unrelated filler stimuli intervened between prime and target. In all experiments, there was a reduction in target-naming errors in the related conditions, and in Experiment 3 this was shown to be largely a reduction in naming failures. The results suggest that related name representations for famous people are not activated in parallel and in competition, and that there is some evidence for a relatively long lasting facilitatory effect. These results require some modification to any serial account of face naming to differentiate it from the generally well-established serial account of object naming.     

The compatibility heuristic in non-categorical hypothetical reasoning: Inferences between conditionals and disjunctions

A new theory explains how people make hypothetical inferences from a premise consistent with several alternatives to a conclusion consistent with several alternatives. The key proposal is that people rely on a heuristic that identifies compatible possibilities. It is tested in 7 experiments that examine inferences between conditionals and disjunctions. Participants accepted inferences between conditionals and inclusive disjunctions when a compatible possibility was immediately available, in their binary judgments that a conclusion followed or not (Experiment 1a) and ternary judgments that included it was not possible to know (Experiment 1b). The compatibility effect was amplified when compatible possibilities were more readily available, e.g., for ‘A only if B’ conditionals (Experiment 2). It was eliminated when compatible possibilities were not available, e.g., for ‘if and only if A B’ bi-conditionals and exclusive disjunctions (Experiment 3). The compatibility heuristic occurs even for inferences based on implicit negation e.g., ‘A or B, therefore if C D’ (Experiment 4), and between universals ‘All A’s are B’s’ and disjunctions (Experiment 5a) and universals and conditionals (Experiment 5b). The implications of the results for alternative theories of the cognitive processes underlying hypothetical deductions are discussed.     

Toddlers’ referential understanding of pictures

Pictures are referential in that they can represent objects in the real world. Here we explore the emergence of understanding of the referential potential of pictures during the second year of life. In Study 1, 15-, 18-, and 24-month-olds learned a word for a picture of a novel object (e.g., “blicket”) in the context of a picture book interaction. Later they were presented with the picture of a blicket along with the real object it depicted and asked to indicate the blicket. Many of the 24-, 18-, and even 15-month-olds indicated the real object as an instance of a blicket, consistent with an understanding of the referential relation between pictures and objects. In Study 2, children were tested with an exemplar object that differed in color from the depicted object to determine whether they would extend the label they had learned for the depicted object to a slightly different category member. The 15-, 18-, and 24-month-old participants failed to make a consistent referential response. The results are discussed in terms of whether pictorial understanding at this age is associative or symbolic.     

Stroop effects on redemption and semantic effects on confession: simultaneous automatic activation of embedded and carrier words

The present study was aimed at assessing whether focusing attention on a task-relevant part of a word prevents processing of its meaning. Participants performed a color-naming task on a prime word followed by lexical decision on a probe. Primes were words, which could contain an embedded color word (e.g., “redemption”) written in an incongruent color. Probes were either semantically related (e.g., “confession”) or unrelated (e.g., “production”) to the prime word. A Stroop effect emerged for color words appearing either in the initial or in the final position of the carrier word. A priming effect also emerged, with faster responses to probes semantically related to the prime. These results are evidence that focusing attention on part of a prime (i.e., the embedded color word) does not prevent the semantic processing of the entire word.