Perceptual and semantic sources of category-specific effects: event-related potentials during picture and word categorization

In two experiments the effect of object category on event-related potentials (ERPs) was assessed while subjects performed superordinate categorizations with pictures and words referring to objects from natural (e.g., animal) and artifactual (e.g., tool) categories. First, a category probe was shown that was presented as name in Experiment 1 and as picture in Experiment 2. Thereafter, the target stimulus was displayed. In both experiments, analyses of the ERPs to the targets revealed effects of category at about 160 msec after target onset in the pictorial modality, which can be attributed to category-specific differences in perceptual processing. Later, between about 300-500 msec, natural and artifactual categories elicited similar ERP effects across target and category modalities. These findings suggest that perceptual as well as semantic sources contribute to category-specific effects. They support the view that semantic knowledge associated with different categories is represented in multiple subsystems that are similarly accessed by pictures and words.     

Semantic priming effects on picture and word processing

The effects of semantic priming on picture and word processing were assessed under conditions in which subjects were required simply to identify stimuli (label pictures or read words) as rapidly as possible. Stimuli were presented in pairs (a prime followed by a target), with half of the pairs containing members of the same semantic category and half containing unrelated concepts. Semantic relatedness was found to facilitate the identification of both pictures (Experiment 1) and words (Experiment 2), and obtained interactions of semantic relatedness and stimulus quality in both experiments suggested that semantic priming affects the initial encoding of both types of stimuli. In Experiment 3, subjects received pairs of pictures, pairs of words, and mixed pairs composed of a picture and a word or of a word and a picture. Significant priming effects were obtained on mixed as well as unmixed pairs, supporting the assumption that pictures and words access semantic information from a common semantic store. Of primary interest was the significantly greater priming obtained in picture-picture pairs than in word-word or mixed pairs. This suggests that, in addition to priming that is mediated by the semantic system, priming may occur in picture-picture pairs that results from the overlap in visual features common to the pictorial representations of objects from the same semantic category.     

Developmental trends in the use of perceptual and conceptual attributes in grouping,clustering, and retrieval

In Experiment I, two tasks were administered to children aged 4, 5, and 9 in order to investigate preference for perceptual versus conceptual attributes in grouping of common objects, and in various aspects of memory. The grouping task revealed a clear chronological progression: color and form determined the youngest children's grouping about equally; form dominated in the 5-year-olds; and most of the oldest children grouped primarily by conceptual attributes. In the memory task, three lists—one organized by color, one by form, one by superordinate category—were presented for free, followed by cued recall. Clustering showed the developmental shift from color to form to concept, while cued recall showed conceptual superiority at all ages. Experiment II replicated the memory task, yielding the same results. The results were discussed in terms of the relative abstractness and predictability of conceptual versus perceptual attributes and the difficulty of abstraction in encoding and the function of predictability in retrieval.     

Conceptual priming with pictures and environmental sounds

A series of experiments was conducted to examine conceptual priming within and across modalities with pictures and environmental sounds. In Experiment 1, we developed a new multimodal stimulus set consisting of two picture and sound exemplars that represented 80 object items. In Experiments 2, we investigated whether categorization of the stimulus items would be facilitated by picture and environmental sound primes that were derived from different exemplars of the target items; and in Experiments 3 and 4, we tested the additional influence on priming when trials were consolidated within a target modality and the inter stimulus interval was lengthened. The results demonstrated that target categorization was facilitated by the advanced presentation of conceptually related exemplars, but there were differences in effectiveness when pictures and sounds appeared as primes.     

The stimulus suffix effect as a memory coding phenomenon

Two short-term memory experiments examined the nature of the stimulus suffix effect on auditory linguistic and nonlinguistic stimulus lists. In Experiment 1, where subjects recalled eight-item digit lists, it was found that a silently articulated digit suffix had the same effect on recall for the last list item as a spoken digit suffix. In Experiment 2, subjects recalled lists of sounds made by inanimate objects either by listing the names of the objects or by ordering a set of drawings of the objects. Auditory suffixes, either another object sound or the spoken name of an object, produced a suffix effect under both recall conditions, but a visually presented picture also produced a suffix effect when subjects recalled using pictures. The results were most adequately explained by a levels-of-processing memory coding hypothesis.     

Facilitation and Inhibition in the Superordinate and Instance Naming of Pictures

A new paradigm, analogous to that used to study inhibition in visual selection (Tipper, 1985), was employed to investigate the possible role of inhibition in selection among instance names (e.g.rabbit) and superordinate names (e.g.animal) for pictured objects. Participants named two sequentially presented pictures in pairs of three types: SAME (e.g., RABBIT-RABBIT), SAME CATEGORY (e.g., HORSE-RABBIT), and DIFFERENT (e.g., BALL-RABBIT). Naming instructions were varied so that participants either used superordinate (S) or instance (I) names for both pictures in a pair (S-S or I-I; Experiment 1) or switched predictably from one type of name to the other across the two pictures (S-I or I-S; Experiment 2). The patterns of naming reaction times revealed both facilitatory and inhibitory effects and provided preliminary support for the hypothesis that selection of a target name involves inhibition of a competing distractor name. Suggestions are given for extending this new paradigm to provide further evidence on the mechanisms underlying name selection.     

Integrating pictorial information across eye movements

Six experiments are reported dealing with the types of information integrated across eye movements in picture perception. A line drawing of an object was presented in peripheral vision, and subjects made an eye movement to it. During the saccade, the initially presented picture was replaced by another picture that the subject was instructed to name as quickly as possible. The relation between the stimulus on the first fixation and the stimulus on the second fixation was varied. Across the six experiments, there was about 100-130 ms facilitation when the pictures were identical compared with a control condition in which only the target location was specified on the first fixation. This finding clearly implies that information about the first picture facilitated naming the second picture. Changing the size of the picture from one fixation to the next had little effect on naming time. This result is consistent with work on reading and low-level visual processes in indicating that pictorial information is not integrated in a point-by-point manner in an integrated visual buffer. Moreover, only about 50 ms of the facilitation for identical pictures could be attributed to the pictures having the same name. When the pictures represented the same concept (e.g., two different pictures of a horse), there was a 90-ms facilitation effect that could have been the result of either the visual or conceptual similarity of the pictures. However, when the pictures had different names, only visual similarity produced facilitation. Moreover, when the pictures had different names, there appeared to be inhibition from the competing names. The results of all six experiments are consistent with a model in which the activation of both the visual features and the name of the picture seen on the first fixation survive the saccade and combine with the information extracted on the second fixation to produce identification and naming of the second picture.     

Processing global information in briefly presented pictures

Summary The effect on object recognition of the processing of the global information in contextually coherent scenes was investigated in two experiments. In Experiment 1 subjects saw 100 ms presentations of line drawings containing objects that were either usual or unusual given the picture context. Following each exposure they were required to select from among four objects the one that had been contained in the scene. The consistency of the three distractor alternatives with the meaning of the picture was varied. Recognition accuracy was poor for unusual objects and when the distractors were consistent with the picture meaning and did not substantially differ from the performance of subjects who selected response alternatives after being provided with a theme of the picture, without actually viewing the pictures. This supports the conclusion that subjects were responding on the basis of the global information but not the local object information in the pictures. When the exposure duration was increased to 2 s (Experiment 2), processing of local information was apparent for both usual and unusual objects but effects of the global information were still evident.     

感知觉加工与概念加工对联结记忆中图片优势效应的影响

本研究采用联结再认范式考察联结记忆中感知觉水平和概念加工程度对图片优势效应的影响。实验1通过呈现清晰或模糊的词语或图片对操纵了感知觉水平,结果发现只有在清晰条件下图片优势效应才会出现;实验2则在模糊条件下通过要求被试想象两个项目之间的关系操纵了概念加工程度,结果发现在有概念加工条件下,出现了图片优势效应。研究结果表明：（1）降低的感知觉水平会导致联结记忆中的图片优势效应消失;（2）对模糊项目对进行概念加工会使联结记忆中出现图片优势效应。     

Revisiting the picture-superiority effect in symbolic comparisons: do pictures provide privileged access?

In 4 experiments, symbolic comparisons were investigated to test semantic-memory retrieval accounts espousing processing advantages for picture over word stimuli. In Experiment 1, participants judged pairs of animal names or pictures by responding to questions probing concrete or abstract attributes (texture or size, ferocity or intelligence). Per pair, attributes were salient or nonsalient concerning their prerated relevance to animals being compared. Distance (near or far) between attribute magnitudes was also varied. Pictures did not significantly speed responding relative to words across all other variables. Advantages were found forfar attribute magnitudes (i.e., the distance effect) and salient attributes. The distance effect was much less for salient than nonsalient concrete-attribute comparisons. These results were consistently found in additional experiments with increased statistical power to detect modality effects. Our findings argue against dual-coding and some common-code accounts of conceptual attribute processing, urging reexamination of the assumption that pictures confer privileged access to long-term knowledge.     

Categorizing chairs and naming pears: Category differences in object processing as a function of task and priming

Four experiments are reported examining the locus of structural similarity effects in picture recognition and naming with normal subjects. Subjects carried out superordinate categorization and naming tasks with picture and word forms of clothing, furniture, fruit, and vegetable exemplars. The main findings were as follows: (1) Responses to pictures of fruit and vegetables (\ldstructurally similar\rd objects) were slowed relative to pictures of clothing and furniture (\ldstructurally dissimilar\rd objects). This structural similarity difference was greater for picture naming than for superordinate categorization of pictures. (2) Structural similarity effects in picture naming were reduced by repetition priming. Repetition priming effects were equivalent from picture and word naming as prime tasks. (3) However, superordinate categorization of the prime did not produce the structural similarity effects on priming found for picture naming. Furthermore, such priming effects did not arise for picture or word categorization or for reading picture names as target tasks. It is proposed that structural similarity effects on priming object processing are located in processes mapping semantic representations of pictures to name representations required to select names for objects. Visually based competition between fruit and vegetables produces competition in name selection, which is reduced by priming the mappings between semantic and name representations.     

Mental comparisons involving abstract attributes

Adult subjects in two experiments were presented pairs of stimuli that differed in varying degree on an abstract semantic attribute, and were required to choose the one with the higher value on the given dimension. Subjects in Experiment 1 chose the more pleasant member of a pair of pictures, concrete nouns, or abstract nouns. Those in Experiment 2, presented a pair of pictures or concrete nouns, chose the one whose referent had the higher monetary value. Theoretical interest centered on the effects of semantic distance, stimulus mode, and individual differences in imagery and verbal ability on choice time. In both experiments, response times (1) decreased with increases in semantic distance, (2) were faster for pictures than words (and for concrete than abstract words in Experiment 1), and (3) were faster for high- than for lowimagery participants. The results are completely consistent with a dual-coding (image vs. verbal) interpretation: Pleasantness and value, though conceptually abstract, are attributes of things rather than words, and they are accordingly represented in and processed by a system specialized for dealing with nonverbal information.     

空间隐喻和形状变化对物体内隐时间概念加工的影响

通过2个实验, 考察了空间隐喻和形状变化对物体内隐时间概念加工的影响。实验1通过对隐含时间关系的词对的语义相关判断发现, 形状变化物体隐含的“先前/后来”的时间概念与“左/右”的空间概念存在着对应关系, 断裂式变化(形状变化大)的物体比渐进式变化(形状变化小)的物体激发了更加明显的时间变化感, 但物体形状变化类型并未明显地影响对词对语义相关判断的速度和准确性。实验2通过对隐含时间关系的物体图片对的语义相关判断发现, 物体形状变化隐含的“先前/后来”的时间概念与“左/右”的空间概念亦存在着对应关系, 而且物体形状变化类型影响对物体图片对语义相关判断的速度和准确性, 被试对渐进式变化物体的语义相关判断显著快于对断裂式变化物体的语义相关判断, 错误率亦低。整个研究表明, 在物体形状变化内隐时间概念的表征中, 既存在着抽象的符号表征, 又存在着具体的形状知觉表征。研究结果支持概念双加工理论的预言。     

The Effects of Prior Processing Episodes on Basic level Superiority

Four experiments were conducted to investigate the effects of prior processing episodes on people's preference for categorizing objects at the basic level (e.g. dog) relative to their preference for categorizing at the superordinate (e.g. animal) and the subordinate (e.g. Dalmation) levels. The prior processing episode in Experiment 1 was designed to induce subjects to activate representations at the superordinate level, and those in the remaining experiments were designed to induce subjects to differentiate objects at the subordinate level. After the prior processing episodes, subjects performed either a free naming or a picture categorization task that required them to decide whether an illustrated object belonged to a specified category. Results showed that prior processing episodes modestly reduced the superiority of basic level to superordinate level and subordinate level in categorization but not in free naming. The results suggest that the basic-level advantage is subject to the effects of context, but the effects are not as strong as the context effects on other aspects of categorization behaviour (e.g. rating typicality of a category member). Hence, the preference for the basic level is a somewhat more stable, invariant aspect of conceptual representation. Possible determinations of this stability are discussed.     

Speech-like coding of pictures in short-term memory

Memory span for pictures of common objects and for the names of these objects was examined as a function of three speech-related variables. Both picture span and name span were found to be influenced by the phonological similarity (Experiment 1) and the length (Experiment 2) of the names, as well as by the subject’s engaging in “irrelevant” vocalization during item presentation (Experiment 3). Moreover, for each variable the effect was in the same direction and of comparable magnitude for the two types of items. Experiments 4-6 replicated these findings with the procedure modified such that the retention of order information was not required. It is concluded that under the present conditions, there is a substantial functional equivalence between short-term memory for readily nameable pictures and for words and that this equivalence may be thought of as due to mediation by a common, “speech-like” code.     

The effects of negative relatedness upon word-picutre and word-word comparisons and subsequent recall.

In two experiments subjects were required to compare the meanings of either a word and a picture, or of two words. Different name levels, i.e. category versus superordinate names, had only a small effect on the time to compare a name with a picture. When incongruent stimulus pairs were semantically related, both positive and negative decision times were longer than when the incongruent pairs were unrelated; relatedness also affected subjects' recall of stimuli. Implications for models of semantic decision are discussed.     

The neural substrates of action retrieval: An examination of semantic and visual routes to action

We report three PET experiments that examine the neural substrates of the conceptual components of action retrieval. In all three experiments, subjects made action or screen-size decisions to familiar objects presented either as pictures or written words (the names ofthe objects). In Experiment 1, a third task was included, requiring a decision on the real-life size of the stimuli. In Experiment 2, a third stimulus type was included, with action and size decisions also performed on pictures of meaningless novel objects. Finally, in Experiment 3, we changed the response mode from a button press to a more explicit movement made with a “manipulandum”. Based on neuropsychological findings, we predicted that when action responses were made to pictures of familiar or novel objects, relative to words, there would be less activation in semantic regions but greater activation in visual, motor, and perhaps parietal cortices. We found that, action relative to screen-size decisions on both pictures and words activated the left hemisphere temporo-frontal semantic system with activation in the left posterior middle temporal cortex specific to action retrieval (Experiment 1). In addition, action retrieval elicited more activation for (1) words than pictures in areas associated with semantics; and (2) novel objects than words or familiar objects in areas associated with pre-semantic object processing. These results are discussed in the context of semantic and visual routes to action retrieval.     

How children view pictures: Children's responses to pictures as things in themselves and as representations of something else

In four experiments we examined under what conditions children around 4 years of age could overcome their normal tendency to treat a picture as transparent and attend to it also as a thing in itself. In Experiment 1, many children chose a picture of food as something “you can really eat” even after having identified it as “just a picture.” In Experiment 2, however, children were able to respond appropriately to implicit requests for a picture as distinct from a real exemplar of the thing represented when the context indicated the correct response. In these tasks, children could succeed by treating picture status simply as an attribute. In Experiments 3 and 4, children were asked to comment on the appearance and reality of pictures, deceptive (trick) objects and real things. Significantly poorer performance on pictures and tricks than on real things suggested that some children have difficulty holding two interpretations of the same input; that is, understanding that a picture has two identities.     

Tracing the time course of picture--word processing

A number of independent lines of research have suggested that semantic and articulatory information become available differentially from pictures and words. The first of the experiments reported here sought to clarify the time course by which information about pictures and words becomes available by considering the pattern of interference generated when incongruent pictures and words are presented simultaneously in a Stroop-like situation. Previous investigators report that picture naming is easily disrupted by the presence of a distracting word but that word naming is relatively immune to interference from an incongruent picture. Under the assumption that information available from a completed process may disrupt an ongoing process, these results suggest that words access articulatory information more rapidly than do pictures. Experiment 1 extended this paradigm by requiring subjects to verify the category of the target stimulus. In accordance with the hypothesis that picture access the semantic code more rapidly than words, there was a reversal in the interference pattern: Word categorization suffered considerable disruption, whereas picture categorization was minimally affected by the presence of an incongruent word. Experiment 2 sought to further test the hypothesis that access to semantic and articulatory codes is different for pictures and words by examining memory for those items following naming or categorization. Categorized words were better recognized than named words, whereas the reverse was true for pictures, a result which suggests that picture naming involves more extensive processing than picture categorization. Experiment 3 replicated this result under conditions in which viewing time was held constant. The last experiment extended the investigation of memory differences to a situation in which subjects were required to generate the superordinate category name. Here, memory for categorized pictures was as good as memory for named pictures. Category generation also influenced memory for words, memory performance being superior to that following a yes--no verification of category membership. These experiments suggest a model of information access whereby pictures access semantic information were readily than name information, with the reverse being true for words. Memory for both pictures and words was a function of the amount of processing required to access a particular type of information as well as the extent of response differentiation necessitated by the task.