Examining lateralized semantic access using pictures

A divided visual field (DVF) experiment examined the semantic processing strategies employed by the cerebral hemispheres to determine if strategies observed with written word stimuli generalize to other media for communicating semantic information. We employed picture stimuli and vary the degree of semantic relatedness between the picture pairs. Participants made an on-line semantic relatedness judgment in response to sequentially presented pictures. We found that when pictures are presented to the right hemisphere responses are generally more accurate than the left hemisphere for semantic relatedness judgments for picture pairs. Furthermore, consistent with earlier DVF studies employing words, we conclude that the RH is better at accessing or maintaining access to information that has a weak or more remote semantic relationship. We also found evidence of faster access for pictures presented to the LH in the strongly-related condition. Overall, these results are consistent with earlier DVF word studies that argue that the cerebral hemispheres each play an important and separable role during semantic retrieval.     

The semantic nature of response competition in the picture-word interference task

Picture-word interference refers to the fact that when a picture (i.e., line drawing) is presented with a word superimposed, picture naming latency is longer than when the picture is presented alone. In addition, naming latency will be further prolonged whenever the word and the picture are members of the same semantic category. This semantic interference effect was investigated in a series of studies in order to develop an appropriate model of the semantic processes involved in picture-word interference. In Experiments 1 and 2, it was determined that the associative strength between the word and the picture is unimportant in the picture-word interference task. In Experiment 3, it was demonstrated that the category typicality of the word and the picture is also unimportant in this task. These results suggest that the semantic processes in picture-word interference would not be well described by a semantic network model. This conclusion was reinforced by Experiment 4, in which it was found that the imageability of a word is a highly important factor in the picture-word interference task. The present set of results suggests that any model of the processes involved in picture-word interference must have at its core the notion of the word’s "relevance" to the task of naming the presented picture.     

汉语词汇产生中的义、音信息提取时间进程的ERP研究

摘 要 运用事件相关电位（ERP）技术考查了在汉语词汇产生过程中,语义提取和语音提取的时间进程。在实验过程中,给被试呈现一张图片,要求他们在小声进行图片命名之前,先基于语义信息（图上画的是物体还是动物）或语音信息（如图片名称是以字母s还是以y开头）完成一个双重选择任务。在一种情况下,反应手（左手或右手按键）由语义信息决定,是否反应（GO/NOGO）由语音信息决定;在另一种情况下,反应手由语音信息决定,是否反应由语义信息决定。结果发现,当是否反应由语义信息（动物/物体）决定时,N200的潜伏期是307ms;当是否反应由语音信息（如中文名称以S/Y开头）决定时,N200的潜伏期是447ms;两者之间存在显著的差异,即由语义信息引起的N200的潜伏期早于由语音信息引起的N200的潜伏期。结果表明,在汉语词汇产生过程中,语义提取先于语音提取。     

Semantic priming over unrelated trials: Evidence for different effects in word and picture naming

Two naming experiments are reported that replicated previous findings of semantic interference as a result of naming related word or picture primes three trials before picture targets. We also examined whether semantic interference occurred when the materials were reversed and picture or word primes were named before word targets. The interest in semantic interference during word naming followed a suggestion made by Humphreys, Lloyd-Jones, and Fias (1995) that word naming, like picture naming, may be reliant on a semantic route to name retrieval when the two stimuli are mixed. In contrast to their findings, we found no evidence for semantic interference during target word naming; in fact, we found facilitation from related picture primes. No priming was found for the related word prime and word target condition. The data allow us to rule out the possibility that word naming is reliant on a semantic route when mixed with pictures in this priming paradigm and to conclude that there is no clear evidence of semantic activation during word naming. We also conclude, in line with other research, that word naming and picture naming involve different processes.     

Bilateral capacity for speech sound processing in auditory comprehension: evidence from Wada procedures

Data from lesion studies suggest that the ability to perceive speech sounds, as measured by auditory comprehension tasks, is supported by temporal lobe systems in both the left and right hemisphere. For example, patients with left temporal lobe damage and auditory comprehension deficits (i.e., Wernicke's aphasics), nonetheless comprehend isolated words better than one would expect if their speech perception system had been largely destroyed (70-80% accuracy). Further, when comprehension fails in such patients their errors are more often semantically-based, than-phonemically based. The question addressed by the present study is whether this ability of the right hemisphere to process speech sounds is a result of plastic reorganization following chronic left hemisphere damage, or whether the ability exists in undamaged language systems. We sought to test these possibilities by studying auditory comprehension in acute left versus right hemisphere deactivation during Wada procedures. A series of 20 patients undergoing clinically indicated Wada procedures were asked to listen to an auditorily presented stimulus word, and then point to its matching picture on a card that contained the target picture, a semantic foil, a phonemic foil, and an unrelated foil. This task was performed under three conditions, baseline, during left carotid injection of sodium amytal, and during right carotid injection of sodium amytal. Overall, left hemisphere injection led to a significantly higher error rate than right hemisphere injection. However, consistent with lesion work, the majority (75%) of these errors were semantic in nature. These findings suggest that auditory comprehension deficits are predominantly semantic in nature, even following acute left hemisphere disruption. This, in turn, supports the hypothesis that the right hemisphere is capable of speech sound processing in the intact brain.     

Is there semantic interference in delayed naming?

The semantic interference effect in the picture-word interference task is interpreted as an index of lexical competition in prominent speech production models. Janssen, Schirm, Mahon, and Caramazza (2008) challenged this interpretation on the basis of experiments with a novel version of this task, which introduced a task-switching component. Participants either named the picture or read the word, depending on the word's color. Janssen et al. reported semantic interference in picture naming, regardless of whether the word appeared simultaneously with the picture (immediate naming) or 1,000 ms after the picture (delayed naming). Because picture name retrieval is completed in less than 1,000 ms, the finding in delayed naming was taken as evidence against the lexical competition account. In 3 sets of experiments conducted in German and English, we tested for semantic effects in Janssen et al.'s task-switching version and in the standard picture-word interference task. Using identical materials, we obtained sizeable interference effects in the standard task (Experiments 2, 4, and 6) but no effects in the task-switching version (Experiments 1, 3, and 5). When the word reading trials of the task-switching version were replaced with no-go trials (Experiment 7), semantic interference reemerged in immediate naming but was still absent in delayed naming. The experiments question the reliability of Janssen et al.'s critical finding and suggest that theoretical inferences about the origin of semantic effects in the standard picture-word interference task based on results from the task-switching version used by Janssen et al. are difficult to draw.     

Semantic facilitation with pictures and words

The present experiments explored the role of processing level and strategic factors in cross-form (word-picture and picture-word) and within-form (picture-picture and word-word) semantic facilitation. Previous studies have produced mixed results. The findings presented in this article indicate that semantic facilitation depends on the task and on the subjects' strategies. When the task required semantic processing of both picture and word targets (e.g., category verification), equivalent facilitation was obtained across all modality combinations. When the task required name processing (e.g., name verification, naming), facilitation was obtained for the picture targets. In contrast, with word targets, facilitation was obtained only when the situation emphasized semantic processing. The results are consistent with models that propose a common semantic representation for both picture and words but that also include assumptions regarding differential order of access to semantic and phonemic features for these stimulus modalities.     

Semantic component of a cross-modal Stroop-like task

Three experiments showed that the pattern of interference of single-modality Stroop tests also exists cross-modally. Distractors and targets were either pictures or auditory words. In a naming task (Experiment 1), word distractors from the same semantic category as picture targets interfered with picture naming more than did semantically unrelated distractors; the semantic category of picture distractors did not differentially affect word naming. In a categorization task (Experiment 2), this Stroop-like effect was reversed: Picture distractors from the same semantic category as word targets interfered less with word categorization than picture distractors that were semantically unrelated; the semantic category of word distractors did not differentially affect picture categorization. Experiment 3 replicated these effects when each subject performed both tasks; the task, naming or categorizing, determined the pattern of interference between pictures and auditory words. The results thus support the existence of a semantic component of a cross-modal Stroop-like effect.     

Semantic guidance of attention within natural scenes

When viewing real-world scenes composed of a myriad of objects, detecting changes can be quite difficult, especially when transients are masked. In general, changes are noticed more quickly and accurately if they occur at the currently (or a recently) attended location. Here, we examine the effects of explicit and implicit semantic cues on the guidance of attention in a change detection task. Participants first attempted to read aloud a briefly presented prime word, then looked for a difference between two alternating versions of a real-world scene. Helpful primes named the object that changed, while misdirecting primes named another (unchanging) object in the picture. Robust effects were found for both explicit and implicit priming conditions, with helpful primes yielding faster change detection times than misdirecting or neutral primes. This demonstrates that observers are able to use higher order semantic information as a cue to guide attention within a natural scene, even when the semantic information is presented outside of explicit awareness.     

Semantic cueing effects on word retrieval in aphasic patients with lexical retrieval deficit

We examined lexical access in word production, focusing on lexical retrieval deficit in aphasic patients. These patients were asked to name one picture at a time and we investigated how semantic cueing affected their ability to retrieve the word. When the subjects experienced a retrieval failure, they were presented with one type of cue, and then they tried to name the target picture again. Four types of cues were used: phonological, semantic related, semantic category member, and baseline cue. Both phonological and semantically related cues improved performance. In contrast, category member cues had no effects on releasing the retrieval block. These results can be explained by an interactive activation model in word production (Dell & O'Seaghdha, 1991).     

The role of visual form in lexical access: Evidence from Chinese classifier production

The interface between the conceptual and lexical systems was investigated in a word production setting. We tested the effects of two conceptual dimensions – semantic category and visual shape – on the selection of Chinese nouns and classifiers. Participants named pictures with nouns (“rope”) or classifier–noun phrases (“one-classifier–rope”) in three blocked picture naming experiments. In Experiment 1, we observed larger semantic category interference with phrases than with nouns, suggesting comparable semantic categorical effects on classifier and noun selection. In Experiments 2 and 3, items with similar shapes produced an interference effect when they were named with classifier–noun phrases, but not with bare nouns. This indicates that object shape modulates classifier (but not noun) selection. We conclude that object shape properties can by themselves influence word selection processes just as semantic relationships (captured by semantic category) do. The factors operating during word selection may be more diverse than has been previously thought.     

The nature of the visual environment induces implicit biases during language-mediated visual search

Four eyetracking experiments examined whether semantic and visual-shape representations are routinely retrieved from printed word displays and used during language-mediated visual search. Participants listened to sentences containing target words that were similar semantically or in shape to concepts invoked by concurrently displayed printed words. In Experiment 1, the displays contained semantic and shape competitors of the targets along with two unrelated words. There were significant shifts in eye gaze as targets were heard toward semantic but not toward shape competitors. In Experiments 2–4, semantic competitors were replaced with unrelated words, semantically richer sentences were presented to encourage visual imagery, or participants rated the shape similarity of the stimuli before doing the eyetracking task. In all cases, there were no immediate shifts in eye gaze to shape competitors, even though, in response to the Experiment 1 spoken materials, participants looked to these competitors when they were presented as pictures (Huettig & McQueen, 2007). There was a late shape-competitor bias (more than 2,500 ms after target onset) in all experiments. These data show that shape information is not used in online search of printed word displays (whereas it is used with picture displays). The nature of the visual environment appears to induce implicit biases toward particular modes of processing during language-mediated visual search.     

Semantic priming effects on picture and word processing

The effects of semantic priming on picture and word processing were assessed under conditions in which subjects were required simply to identify stimuli (label pictures or read words) as rapidly as possible. Stimuli were presented in pairs (a prime followed by a target), with half of the pairs containing members of the same semantic category and half containing unrelated concepts. Semantic relatedness was found to facilitate the identification of both pictures (Experiment 1) and words (Experiment 2), and obtained interactions of semantic relatedness and stimulus quality in both experiments suggested that semantic priming affects the initial encoding of both types of stimuli. In Experiment 3, subjects received pairs of pictures, pairs of words, and mixed pairs composed of a picture and a word or of a word and a picture. Significant priming effects were obtained on mixed as well as unmixed pairs, supporting the assumption that pictures and words access semantic information from a common semantic store. Of primary interest was the significantly greater priming obtained in picture-picture pairs than in word-word or mixed pairs. This suggests that, in addition to priming that is mediated by the semantic system, priming may occur in picture-picture pairs that results from the overlap in visual features common to the pictorial representations of objects from the same semantic category.     

Can residual lexical knowledge concern word form rather than word meaning?

We described a patient with a dramatic deficit of both word comprehension and naming but with good preservation of visual pictorial semantics. On word-picture matching, his performances were slightly better than expected based on the observed lexical semantic disorder; in addition, the patient, who maintained good preservation of his underlying phonology, showed a tendency to point to the picture phonologically related to the target. In order to interpret these data, we advanced the hypothesis that the patient, in spite of his virtually complete inability to name, would be able, in a word-picture matching task, to "covertly" (i.e., preverbally) retrieve the name from the picture and to use this name to attempt a match with the phonological form of the stimulus word. This mechanism, that we called "phonological" comprehension, would allow the identification of the correct target and would explain the choice of the phonologically related foil that was sometimes selected.     

Assessing the influence of schematic drawings of body parts on tactile discrimination performance using the crossmodal congruency task

Seeing one's own body (either directly or indirectly) can influence visuotactile crossmodal interactions. Recently, it has been shown that even viewing a simple line drawing of a hand can also modulate such crossmodal interactions, as if the picture of the hand somehow corresponds to (or primes) the participants' own hand. Alternatively, however, it could be argued that the modulatory effects of viewing the picture of a hand on visuotactile interactions might simply be attributed to cognitive processes such as the semantic referral to the relevant body part or to the orientation cues provided by the hand picture instead. In the present study, we evaluated these various different interpretations of the hand picture effect. Participants made speeded discrimination responses to the location of brief vibrotactile targets presented to either the tip or base of their forefinger, while trying to ignore simultaneously-presented visual distractors presented to either side of central fixation. We compared the modulatory effect of the picture of a hand with that seen when the visual distractors were presented next to words describing the tip and base of the forefinger (Experiment 1), or were superimposed over arrows which provided another kind of directional cue (Experiment 2). Tactile discrimination performance was modulated in the hand picture condition, but not in the word or arrow conditions. These results therefore suggest that visuotactile interactions are specifically modulated by the image of the hand rather than by cognitive cues such as simply semantic referral to the relevant body sites and/or any visual orientation cues provided by the picture of a hand.     

Semantic classification of pictures and words

We provide new behavioural norms for semantic classification of pictures and words. The picture stimuli are 288 black and white line drawings from the International Picture Naming Project ([Székely, A., Jacobsen, T., D'Amico, S., Devescovi, A., Andonova, E., Herron, D., et al. (2004). A new on-line resource for psycholinguistic studies. Journal of Memory & Language, 51, 247–250]). We presented these pictures for classification in a living/nonliving decision, and in a separate version of the task presented the corresponding word labels for classification. We analyzed behavioural responses to a subset of the stimuli in order to explore questions about semantic processing. We found multiple semantic richness effects for both picture and word classification. Further, while lexical-level factors were related to semantic classification of words, they were not related to semantic classification of pictures. We argue that these results are consistent with privileged semantic access for pictures, and point to ways in which these data could be used to address other questions about picture processing and semantic memory.     

Task choice and semantic interference in picture naming

Evidence from dual-task performance indicates that speakers prefer not to select simultaneous responses in picture naming and another unrelated task, suggesting a response selection bottleneck in naming. In particular, when participants respond to tones with a manual response and name pictures with superimposed semantically related or unrelated distractor words, semantic interference in naming tends to be constant across stimulus onset asynchronies (SOAs) between the tone stimulus and the picture–word stimulus. In the present study, we examine whether semantic interference in picture naming depends on SOA in case of a task choice (naming the picture vs reading the word of a picture–word stimulus) based on tones. This situation requires concurrent processing of the tone stimulus and the picture–word stimulus, but not a manual response to the tones. On each trial, participants either named a picture or read aloud a word depending on the pitch of a tone, which was presented simultaneously with picture–word onset or 350 ms or 1000 ms before picture–word onset. Semantic interference was present with tone pre-exposure, but absent when tone and picture-word stimulus were presented simultaneously. Against the background of the available studies, these results support an account according to which speakers tend to avoid concurrent response selection, but can engage in other types of concurrent processing, such as task choices.     

延迟命名任务中图词干扰效应的眼动研究

采用图-词分离呈现及眼动技术探讨在延时命名条件下图-词干扰效应发生的位置。结果发现：（1）延时命名图片时,仍出现图-词干扰效应和频率效应;（2）与命名干扰词相比,命名图片时图片的回视率高,注视时间长;（3）图片的注视时间也出现频率效应和图-词干扰效应。这说明即使在延时命名图片的条件下,目标词汇仍需要再次激活,频率效应和图-词干扰效应是发生在词汇化过程中,反驳反应排除理论的图-词干扰反应选择阶段的观点,支持了词汇化选择竞争的理论。     

Semantic interference and associative facilitation from words presented in rapid serial visual presentation

Semantic interference in picture naming is readily obtained when categorically related distractor words are displayed with picture targets; however, this is not typically the case when both primes and targets are words. Researchers have argued that to obtain semantic interference for word primes and targets, the prime must be shown for a sufficient duration, prime processing must be made difficult, and participants must attend to the primes. In this article, we used a novel procedure for prime presentation to investigate semantic interference in word naming. Primes were presented as the last word of a rapid serial visual presentation stream, with the target following 600-1,200 msec later. Semantic interference was observed for categorically related targets, whereas facilitation was found for associatively related targets.     

Semantic interference in picture naming during dual-task performance does not vary with reading ability

Previous dual-task studies examining the locus of semantic interference of distractor words in picture naming have obtained diverging results. In these studies, participants manually responded to tones and named pictures while ignoring distractor words (picture–word interference, PWI) with varying stimulus onset asynchrony (SOA) between tone and PWI stimulus. Whereas some studies observed no semantic interference at short SOAs, other studies observed effects of similar magnitude at short and long SOAs. The absence of semantic interference in some studies may perhaps be due to better reading skill of participants in these than in the other studies. According to such a reading-ability account, participants’ reading skill should be predictive of the magnitude of their interference effect at short SOAs. To test this account, we conducted a dual-task study with tone discrimination and PWI tasks and measured participants’ reading ability. The semantic interference effect was of similar magnitude at both short and long SOAs. Participants’ reading ability was predictive of their naming speed but not of their semantic interference effect, contrary to the reading ability account. We conclude that the magnitude of semantic interference in picture naming during dual-task performance does not depend on reading skill.