Verbal labeling as an assimilation mnemonic for abstract visual stimuli: The sample case of recognition memory for Chinese characters

In four experiments, we examined the effect that presenting a verbal stimulus (viz., an English noun) alongside an abstract visual stimulus (viz., a Chinese character) enhances recognition memory for the abstract visual stimulus. Experiment 1 demonstrated that the character-plus-word combination at both encoding and retrieval results in better recognition than does a character-alone presentation or presenting the combination at encoding only. Experiment 2 demonstrated that presenting the word first and then the character results in better performance than does the opposite order. Experiment 3 showed that the concreteness value of the word, not familiarity, is the critical factor. In Experiment 4, presentation time was varied. More time was needed for liftoff from chance level for the word-character combination than for the character-alone presentation. Together, the results suggest that subjects spontaneously assimilate stimulus and word into a single representation by building asymmetric effortful imagery associations, going from the English word to the Chinese character.     

Decomposition of repetition priming processes in word translation

Translation in fluent bilinguals requires comprehension of a stimulus word and subsequent production, or retrieval and articulation, of the response word. Four repetition-priming experiments with Spanish–English bilinguals (N = 274) decomposed these processes using selective facilitation to evaluate their unique priming contributions and factorial combination to evaluate the degree of process overlap or dependence. In Experiment 1, symmetric priming between semantic classification and translation tasks indicated that bilinguals do not covertly translate words during semantic classification. In Experiments 2 and 3, semantic classification of words and word-cued picture drawing facilitated word-comprehension processes of translation, and picture naming facilitated word-production processes. These effects were independent, consistent with a sequential model and with the conclusion that neither semantic classification nor word-cued picture drawing elicits covert translation. Experiment 4 showed that 2 tasks involving word-retrieval processes--written word translation and picture naming--had subadditive effects on later translation. Incomplete transfer from written translation to spoken translation indicated that preparation for articulation also benefited from repetition in the less-fluent language.     

The role of crossmodal competition and dimensional overlap in crossmodal attention switching

Crossmodal selective attention was investigated in a cued task switching paradigm using bimodal visual and auditory stimulation. A cue indicated the imperative modality. Three levels of spatial S–R associations were established following perceptual (location), structural (numerical), and conceptual (verbal) set-level compatibility. In Experiment 1, participants switched attention between the auditory and visual modality either with a spatial-location or spatial-numerical stimulus set. In the spatial-location set, participants performed a localization judgment on left vs. right presented stimuli, whereas the spatial-numerical set required a magnitude judgment about a visually or auditorily presented number word. Single-modality blocks with unimodal stimuli were included as a control condition. In Experiment 2, the spatial-numerical stimulus set was replaced by a spatial-verbal stimulus set using direction words (e.g., “left”). RT data showed modality switch costs, which were asymmetric across modalities in the spatial-numerical and spatial-verbal stimulus set (i.e., larger for auditory than for visual stimuli), and congruency effects, which were asymmetric primarily in the spatial-location stimulus set (i.e., larger for auditory than for visual stimuli). This pattern of effects suggests task-dependent visual dominance.     

The locus of taboo context effects in picture naming

Speakers respond more slowly when naming pictures presented with taboo (i.e., offensive/embarrassing) than with neutral distractor words in the picture–word interference paradigm. Over four experiments, we attempted to localize the processing stage at which this effect occurs during word production and determine whether it reflects the socially offensive/embarrassing nature of the stimuli. Experiment 1 demonstrated taboo interference at early stimulus onset asynchronies of ?150?ms and 0?ms although not at 150?ms. In Experiment 2, taboo distractors sharing initial phonemes with target picture names eliminated the interference effect. Using additive factors logic, Experiment 3 demonstrated that taboo interference and phonological facilitation effects do not interact, indicating that the two effects originate at different processing levels within the speech production system. In Experiment 4, interference was observed for masked taboo distractors, including those sharing initial phonemes with the target picture names, indicating that the effect cannot be attributed to a processing level involving responses in an output buffer. In two of the four experiments, the magnitude of the interference effect correlated significantly with arousal ratings of the taboo words. However, no significant correlations were found for either offensiveness or valence ratings. These findings are consistent with a locus for the taboo interference effect prior to the processing stage responsible for word form encoding. We propose a pre-lexical account in which taboo distractors capture attention at the expense of target picture processing due to their high arousal levels.     

Automatic and directed search processes in solving simple semantic-memory problems

The cognitive processes involved in simple semantic-memory problems were investigated in four experiments. On each trial of Experiments 1 and 2, two stimulus words were presented, with the instructions to find a third word (i.e., the solution) that, when coupled with each of the stimuli, would yield two word pairs used in everyday language (e.g., surprise and birthday, for which the solution is party). The results of the two experiments indicated that informing the subject whether the solution constituted the first or the second element in the word pairs facilitated both likelihood and speed of solution attainment. In addition, solution attainment was relatively high for items based on frequently used word pairs (Experiment 1) and for items in which the stimuli appear, in everyday language, in a small number of word pairs (Experiment 2). In Experiment 3, the subjects were required to produce word pairs containing one of the two stimulus words from the items used in Experiment 2. Solution production was facilitated by rehearsing the second stimulus word of the specific item. The conclusion, supported by a post hoc analysis of the results of Experiments 2 and 3 (Experiment 4), was that indirect priming from one stimulus word may facilitate solution production from a searched word. These results are interpreted in terms of automatic and controlled processes, and their relevance to two different models for retrieval from semantic memory is discussed.     

What are human express saccades?

When a fixation point is removed 200 msec prior to target onset (the gap condition), human subjects are said to produce eye movements that have a short latency (80–120 msec), that form the early peak of a bimodal latency distribution, and that have been labeled “human express saccades” (see, e.g., Fischer, 1987; Fischer & Breitmeyer, 1987; Fischer & Ramsperger, 1984, 1986). In three experiments, we sought to obtain this express saccade diagnostic pattern in the gap condition, We orthogonally combined target location predictability with the presence versus absence of catch trials (Experiment 1). When target location was fixed and catch trials were not used, we found mostly anticipations. In the remaining conditions, where responses were under stimulus control, bimodality was not frequently observed, and, whether it was or not, latencies were not in the express saccade range. Using random target locations, we then varied stimulus luminance and the mode of stimulus presentation (LEDs vs. oscilloscope) in the gap and overlap (fixation is not removed) conditions (Experiment2). Bimodality was rarely observed, the gap effect (overlap minus gap reaction time) was additive with luminance, and only the brightest targets elicited saccades in the express range. When fixed locations and no catch trials were combined with latency feedback (Experiment 3), we observed many responses in the express saccade range and some evidence for bimodality, but the sudden introduction of catch trials revealed that many early responses were not under stimulus control. Humanscan make stimulus-controlled saccades that are initiated very rapidly (80–120 msec), but unless catch trials or choice reaction time is used, it is not possible to distinguish such saccades from anticipatory responses that are prepared in advance and timed to occur shortly after target onset. Because the express saccade diagnostic pattern is not a characteristic feature of human saccadic performance, we urge investigators to focus their attention on the robustgap effect     

On the Functional Equivalence of Monolinguals and Bilinguals in "Monolingual Mode": The Bilingual Anticipation Effect in Picture-Word Processing

Previous evidence indicates that bilinguals are slowed when an unexpected language switch occurs when they are reading aloud. This anticipation effect was investigated using a picture-word translation task to compare English monolinguals and Spanish-English bilinguals functioning in "monolingual mode." Monolinguals and half of the bilinguals drew pictures or wrote English words for picture or English word stimuli; the remaining bilinguals drew pictures or wrote Spanish words for picture or Spanish word stimuli. Production onset latency was longer in cross-modality translation than within-modality copying, and the increments were equivalent between groups across stimulus and production modalities. Assessed within participants, bilinguals were slower than monolinguals under intermixed but not under blocked trial conditions. Results indicate that the bilingual anticipation effect is not specific to language-mixing tasks. More generally, stimulus-processing uncertainty prevents establishment of a "base" symbolic-system procedure (concerning recognition, production, and intervening translation) and the inhibition of others. When this uncertainty is removed, bilinguals exhibit functional equivalence to monolinguals.     

Effect of classical conditioning on an internal clock

Six experiments with rats used a psychophysical choice procedure to study the internal clock used to discriminate duration. They investigated if the clock is sensitive to the signal value (associative strength) of a stimulus. The experiments involved two types of trials. On choice trials, a stimulus lasted a short (e.g., 3 s) or long (e.g., 12 s) duration; then the rats chose between two levers. The rewarded choice depended on the duration of the stimulus. On conditioning trials, the stimulus used on choice trials was presented, but it ended without food (extinction trials) or with food (pairing trials) regardless of what the rat did. The main stimulus minus accuracy with the long stimulus. Experiment 1 showed that extinction trials increased short bias relative to training without conditioning trials or to training with pairing trials. The rest of the experiments tested explanations of these results. The same results were found when extinction trials were the same duration as the short stimulus (Experiment 2), when extinction trials were a random duration (Experiment 5), and when the signal value of the conditioned stimulus was changed in another way (Experiment 6). The effect of conditioning trials was modality specific (Experiments 3 and 4). Of the explanations considered, the best one--the only one not contradicted at least once--is that changing the signal value of a stimulus changes how the clock times the stimulus. Reducing signal value reduces the measured duration.     

The semantic nature of response competition in the picture-word interference task

Picture-word interference refers to the fact that when a picture (i.e., line drawing) is presented with a word superimposed, picture naming latency is longer than when the picture is presented alone. In addition, naming latency will be further prolonged whenever the word and the picture are members of the same semantic category. This semantic interference effect was investigated in a series of studies in order to develop an appropriate model of the semantic processes involved in picture-word interference. In Experiments 1 and 2, it was determined that the associative strength between the word and the picture is unimportant in the picture-word interference task. In Experiment 3, it was demonstrated that the category typicality of the word and the picture is also unimportant in this task. These results suggest that the semantic processes in picture-word interference would not be well described by a semantic network model. This conclusion was reinforced by Experiment 4, in which it was found that the imageability of a word is a highly important factor in the picture-word interference task. The present set of results suggests that any model of the processes involved in picture-word interference must have at its core the notion of the word’s "relevance" to the task of naming the presented picture.     

Perceptual comparisons with laterally presented pictures and environmental sounds

Two experiments studied perceptual comparisons with cues that vary in one of four ways (picture, sound, spoken word, or printed word) and with targets that are either pictures or environmental sounds. The basic question probed whether modality or differences in format were factors that would influence picture and sound perception. Also of interest were cue effect differences when targets are presented on either the right or left side. Students responded to a same-different reaction time task that entailed matching cue-target pairs to determine whether the successive stimulus events represented features drawn from the same basic item. Cue type influenced reaction times to pictures and environmental sounds, but the effects were qualified by response type and with picture targets by presentation side. These results provide some additional evidence of processing asymmetry when pictures are directed to either the right or left hemisphere, as well as for some asymmetries in cross-modality cuing. Implications of these findings for theories of multisensory processing and models of object recognition are discussed.     

Activation of semantic information at the sublexical level during handwriting production: Evidence from inhibition effects of Chinese semantic radicals in the picture‐word interference paradigm

Using a non‐alphabetic language (e.g., Chinese), the present study tested a novel view that semantic information at the sublexical level should be activated during handwriting production. Over 80% of Chinese characters are phonograms, in which semantic radicals represent category information (e.g., 椅 ‘chair,’ 桃 ‘peach,’ 橙 ‘orange’ are related to plants) while phonetic radicals represent phonetic information (e.g., 狼 ‘wolf,’ 朗 ‘brightness,’ 郎 ‘male,’ are all pronounced /lang/). Under different semantic category conditions at the lexical level (semantically related in Experiment 1; semantically unrelated in Experiment 2), the orthographic relatedness and semantic relatedness of semantic radicals in the picture name and its distractor were manipulated under different SOAs (i.e., stimulus onset asynchrony, the interval between the onset of the picture and the onset of the interference word). Two questions were addressed: (1) Is it possible that semantic information could be activated in the sublexical level conditions? (2) How are semantic and orthographic information dynamically accessed in word production? Results showed that both orthographic and semantic information were activated under the present picture‐word interference paradigm, dynamically under different SOAs, which supported our view that discussions on semantic processes in the writing modality should be extended to the sublexical level. The current findings provide possibility for building new orthography‐phonology‐semantics models in writing.     

Semantic priming effects on picture and word processing

The effects of semantic priming on picture and word processing were assessed under conditions in which subjects were required simply to identify stimuli (label pictures or read words) as rapidly as possible. Stimuli were presented in pairs (a prime followed by a target), with half of the pairs containing members of the same semantic category and half containing unrelated concepts. Semantic relatedness was found to facilitate the identification of both pictures (Experiment 1) and words (Experiment 2), and obtained interactions of semantic relatedness and stimulus quality in both experiments suggested that semantic priming affects the initial encoding of both types of stimuli. In Experiment 3, subjects received pairs of pictures, pairs of words, and mixed pairs composed of a picture and a word or of a word and a picture. Significant priming effects were obtained on mixed as well as unmixed pairs, supporting the assumption that pictures and words access semantic information from a common semantic store. Of primary interest was the significantly greater priming obtained in picture-picture pairs than in word-word or mixed pairs. This suggests that, in addition to priming that is mediated by the semantic system, priming may occur in picture-picture pairs that results from the overlap in visual features common to the pictorial representations of objects from the same semantic category.     

Crossmodal temporal discrimination: assessing the predictions of a general pacemaker-counter model

In this study, an extended pacemaker-counter model was applied to crossmodal temporal discrimination. In three experiments, subjects discriminated between the durations of a constant standard stimulus and a variable comparison stimulus. In congruent trials, both stimuli were presented in the same sensory modality (i.e., both visual or both auditory), whereas in incongruent trials, each stimulus was presented in a different modality. The model accounts for the finding that temporal discrimination depends on the presentation order of the sensory modalities. Nevertheless, the model fails to explain why temporal discrimination was much better with congruent than with incongruent trials. The discussion considers possibilities to accommodate the model to this and other shortcomings.     

Holding a manual response sequence in memory can disrupt vocal responses that share semantic features with the manual response

Holding an action plan in memory for later execution can delay execution of another action if the actions share a similar (compatible) feature. This compatibility interference (CI) occurs for actions that share the same response modality (e.g., manual response). We investigated whether CI can generalize to actions that utilize different response modalities (manual and vocal). In three experiments, participants planned and withheld a sequence of key-presses with the left- or right-hand based on the visual identity of the first stimulus, and then immediately executed a speeded, vocal response (‘left’ or ‘right’) to a second visual stimulus. The vocal response was based on discriminating stimulus color (Experiment 1), reading a written word (Experiment 2), or reporting the antonym of a written word (Experiment 3). Results showed that CI occurred when the manual response hand (e.g., left) was compatible with the identity of the vocal response (e.g., ‘left’) in Experiment 1 and 3, but not in Experiment 2. This suggests that partial overlap of semantic codes is sufficient to obtain CI unless the intervening action can be accessed automatically (Experiment 2). These findings are consistent with the code occupation hypothesis and the general framework of the theory of event coding (Behav Brain Sci 24:849–878, 2001a; Behav Brain Sci 24:910–937, 2001b).     

Illusory conjunctions of forms, objects, and scenes during rapid serial visual search

"Temporal migration" describes a situation in which subjects viewing rapidly presented stimuli (e.g., 9-20 items/s) confidently report a target element as having been presented in the same display as a previous or following stimulus in the sequence. Four experiments tested a short-term buffer model of this phenomenon. Experiments 1 and 4 tested the hypothesis that subjects' errors are due to the demands of the verbal report procedure rather than to perceptual integration. In Experiment 1, 12 color objects were presented at a rate of 9/s. Prior to each sequence, an object was named and subjects responded "yes" or "no" to indicate whether the target element (a black frame) occurred with that object. Consistent with the perceptual hypothesis, the yes/no procedure yielded the same results as the verbal report procedure. Experiment 2 tested the hypothesis that the direction of migration depends on "frame" detection time. Results showed that reaction time to frame detection was significantly faster in trials in which subjects reported the frame on a preceding rather than a following picture. Experiments 3 and 4 used the standard naming procedure and the yes/no procedure to test temporal migration using more complex, interrelated stimuli (objects and scenes). Implications for the use of the temporal migration effect to study visual integration within eye fixations are discussed.     

Effects of working memory load on long-term word priming

Three experiments assessed the role of verbal and visuo-spatial working memory in supporting long-term repetition priming for written words. In Experiment 1, two priming tasks (word stem completion and category-exemplar production) were included with three levels of load on working memory: (1) without memory load, (2) memory load that involved storing a string of six digits, and (3) memory load that involved storing a graphic shape. Experiments 2 and 3 compared the effects of a verbal (Experiment 2) or a visual (Experiment 3) working memory load at encoding on both an implicit (word stem completion) and an explicit test (cued recall). The results show no effect of memory load in any of the implicit memory tests, suggesting that priming does not rely on working memory resources. By contrast, loading working memory at encoding causes a significant disruptive effect on the explicit memory test for words when the load is verbal but not visual.     

Comparing color-word and picture-word Stroop-like effects: A test of the Glaser and Glaser (1989) model

Summary Glaser and Glaser (1989) assume that the processing of colors and pictures is highly similar in that, compared to words, both kinds of stimulis have privileged access to semantic information. This assumption was tested in the present research. In Experiment 1, the season corresponding to the color or to the word of color-word Stroop stimuli had to be named (e.g., green for spring). In Experiment 2, subjects had to name the season corresponding to the picture or the word of a picture-word stimulus (e. g., flower for spring). According to Glaser and Glaser (1989), privileged semantic processing of colors and pictures should be evidenced by a larger interfering power of color and picture distractors than of word distractors. However, the asymmetric pattern of interference was observed only with picture-word stimuli (Experiment 2), but not with color-word stimuli (Experiment 1), suggesting that, unlike pictures, colors do not have privileged access to semantic information. It was also found that word distractors interfered with the semantic processing of pictures, a result that is incompatible with the dominance rule postulated by Glaser and Glaser (1989). From these results, an adapted version of the Glaser and Glaser model is proposed: colors are assumed to have privileged access to a separate color processing system and the pattern of interference depends upon the relative activation strength of the response alternatives activated by the target and the distractor.     

The representation of nonstructural information in visual memory: Evidence from image combination

Two experiments investigated the differential representation of the figure and ground of a picture in visual short-term and long-term memory. It is known (Hitch, Brandimonte, & Walker, 1995) that subjects find it more difficult to combine mental images of two separately presented pictures in order to identify a novel form when the two pictures are incongruent in color (i.e., when a black-on-white line drawing has to be combined with a white-on-black drawing). In the present experiments, thefigures were depicted in solid form to allow color congruity to be varied independently for figure and ground. Results showed a clear impairment in image combination when the to-be-combined figures were incongruent in color (black-on-gray and white-on-gray) but not when theirgrounds were incongruently colored (gray-on-black and gray-on-white). In this way, image combination was seen to be supported by a representation of the object depicted in the picture rather than by a literal representation of the picture itself (i.e., a pictorial code). In line with previous findings, the same representation was seen to support image combination based on short-term memory (Experiment 1) and long-term memory (Experiment 2), provided that in the latter case verbal recoding was precluded. When verbal recoding was allowed, image combination based on long-term memory was insensitive to color congruity, implying the involvement of a more abstract structural representation.     

The role of sensory-motor modality compatibility in language processing

Language processing requires the combination of compatible (auditory-vocal and visual-manual) or incompatible (auditory-manual and visual-vocal) sensory-motor modalities, and switching between these sensory-motor modality combinations is very common in every-day life. Sensory-motor modality compatibility is defined as the similarity of stimulus modality and the modality of response-related sensory consequences. We investigated the influence of sensory-motor modality compatibility during performing language-related cognitive operations on different linguistic levels. More specifically, we used a variant of the task-switching paradigm, in which participants had to switch between compatible or between incompatible sensory-motor modality combinations during a verbal semantic categorization (Experiment 1) or during a word-form decision (Experiment 2). The data show higher switch costs (i.e., higher reaction times and error rates in switch trials compared to repetition trials) in incompatible sensory-motor modality combinations than in compatible sensory-motor modality combinations. This was true for every language-related cognitive operation, regardless of the individual linguistic level. Taken together, the present study demonstrates that sensory-motor modality compatibility plays an important role in modality switching during language processing.     

Transfer across Form and Modality in Implicit and Explicit Memory

Three experiments examined transfer across form (words/pictures) and modality (visual/ auditory) in written word, auditory word, and pictorial implicit memory tests, as well as on a free recall task. Experiment 1 showed no significant transfer across form on any of the three implicit memory tests,and an asymmetric pattern of transfer across modality. In contrast, the free recall results revealed a very different picture. Experiment 2 further investigated the asymmetric modality effects obtained for the implicit memory measures by employing articulatory suppression and picture naming to control the generation of phonological codes. Finally, Experiment 3 examined the effects of overt word naming and covert picture labelling on transfer between study and test form. The results of the experiments are discussed in relation to Tulving and Schacter's (1990) Perceptual Representation Systems framework and Roediger's (1990) Transfer Appropriate Processing theory.