The role of visual similarity in picture categorization

Categorization is usually assumed to require access to a concept's meaning. When pictures are categorized faster than words, they are assumed to be understood faster than words. However, pictures from the same category are more similar than pictures from different categories. The present article argues that the use of visual similarity as a cue to category membership may produce the picture advantage. The visual similarity hypothesis was tested in two experiments. In the first experiment, pictures showed a disadvantage for the visually similar categories of fruits and vegetables, but showed their usual advantage for the visually dissimilar categories of fruits and animals. In the second experiment, with a mixed list design, pictures were slower only for visually similar different decisions, but showed the usual advantage for all other decisions. The reliability of visual similarity as a cue to the decision accounted well for these results. Because visual similarity can be shown to have large effects on picture categorization, the use of categorization to compare speed of understanding of pictures and words is questionable.     

The picture superiority effect in categorization: visual or semantic?

Two experiments are reported whose aim was to replicate and generalize the results presented by Snodgrass and McCullough (1986) on the effect of visual similarity in the categorization process. For pictures, Snodgrass and McCullough's results were replicated because Ss took longer to discriminate elements from 2 categories when they were visually similar than when they were visually dissimilar. However, unlike Snodgrass and McCullough, an analogous increase was also observed for word stimuli. The pattern of results obtained here can be explained most parsimoniously with reference to the effect of semantic similarity, or semantic and visual relatedness, rather than to visual similarity alone.     

Semantic priming effects on picture and word processing

The effects of semantic priming on picture and word processing were assessed under conditions in which subjects were required simply to identify stimuli (label pictures or read words) as rapidly as possible. Stimuli were presented in pairs (a prime followed by a target), with half of the pairs containing members of the same semantic category and half containing unrelated concepts. Semantic relatedness was found to facilitate the identification of both pictures (Experiment 1) and words (Experiment 2), and obtained interactions of semantic relatedness and stimulus quality in both experiments suggested that semantic priming affects the initial encoding of both types of stimuli. In Experiment 3, subjects received pairs of pictures, pairs of words, and mixed pairs composed of a picture and a word or of a word and a picture. Significant priming effects were obtained on mixed as well as unmixed pairs, supporting the assumption that pictures and words access semantic information from a common semantic store. Of primary interest was the significantly greater priming obtained in picture-picture pairs than in word-word or mixed pairs. This suggests that, in addition to priming that is mediated by the semantic system, priming may occur in picture-picture pairs that results from the overlap in visual features common to the pictorial representations of objects from the same semantic category.     

Development of visual,name, and conceptual memory codes for pictures

In a recognition memory task, 4-, 8-, and 16-year-olds quickly decided whether or not probe pictures were identical to remembered study pictures. Conceptual and visual, but not acoustic, confusion were evidenced by the longer latencies when probes were visually or conceptually related to study items than when unrelated or rhyming. Visual confusion arose only when seven or fewer pictures intervened between study and probe items, and the kinds of visual similarity that produced confusion changed with age. Conceptual confusion resulted only at longer lags, and the same kinds of conceptual relationships produced confusion at all ages. These findings are discussed in terms of the development of visual and semantic encoding processes.     

Searching for politicians: An information-processing approach to face recognition

Previous work on face recognition has concentrated on the processing of unfamiliar faces. This paper examines the recognition of already familiar faces, specifically politicians. Three experiments are described in which the subject's task was to search through a series of faces for particular target politicians. In Experiment I the function relating search time to target set size was found to be negatively accelerated. A similar function was observed when names were used as search items. In Experiment II all subjects searched for four targets, and the relationship between distractors and target items was varied. Distractors rated visually similar to the targets took longer to reject than those rated dissimilar. Distractors who were other politicians took longer to reject than actors, and this effect of semantic category was independent of visual similarity. In Experiment III, where subjects searched for a single target, semantic category appeared only to have an effect when the distractors were also visually similar to the target. Models of the rejection process are discussed, and the similarities between the effects observed here, with faces, and those reported elsewhere for words are pointed out.     

语义相似性与类别大小对图-词干扰范式下语义效应的影响

采用基本水平命名和类别水平命名任务, 通过2个实验, 考察了图-词之间的语义相似性与类别大小对图-词干扰范式下语义效应的影响。实验1发现, 图片命名时间在语义相似性高的干扰词条件下显著快于在语义相似性低的干扰词条件下, 但这种效应只出现在基本水平命名任务中, 未出现在类别水平命名任务中。图-词之间的语义相似性高低对低熟悉图片命名影响大, 对高熟悉图片命名影响小。实验2显示, 语义关联的干扰词对图片的基本水平命名产生了语义干扰效应, 对图片的类别水平命名产生了语义促进效应。类别大小强化了语义效应：类别大, 图片的基本水平命名的语义干扰效应更大, 图片的类别水平命名的语义促进效应更强。整个研究表明, 图-词之间的语义关联的强度影响语义效应的方向, 图-词之间的语义关联的广度影响语义效应的强弱。需要结合多种理论来解释图-词之间的语义相似性和类别大小对图-词干扰范式下语义效应的影响。     

汉语短时序列回忆中的语音相似性和视觉相似性效应

语音相似性效应和视觉相似性效应是短时序列回忆中的两个典型性效应, 但前人很少探讨这两种效应的交互作用。本研究利用汉语字形和语音的属性, 观察汉字短时序列回忆中语音相似性效应、视觉相似性效应及两者的交互作用。研究结果发现当回忆项目在语音或视觉上单纯相似或不相似时, 视觉相似性效应及其与语音相似性的交互作用与混合词表条件下所发现的结果有很大的不同。这个研究结果为丰富和补充相关理论模型提供了进一步实证依据。     

Theoretical analysis of the cognitive processing of lexical and pictorial stimuli: reading, naming, and visual and conceptual comparisons

This article reviews the research literature on the differences between word reading and picture naming. A theory for the visual and cognitive processing of pictures and words is then introduced. The theory accounts for slower naming of pictures than reading of words. Reading aloud involves a fast, grapheme-to-phoneme transformation process, whereas picture naming involves two additional processes: (a) determining the meaning of the pictorial stimulus and (b) finding a name for the pictorial stimulus. We conducted a reading-naming experiment, and the time to achieve (a) and (b) was determined to be approximately 160 ms. On the basis of data from a second experiment, we demonstrated that there is no significant difference in time to visually compare two pictures or two words when size of the stimuli is equated. There is no difference in time to make the two types of cross-modality conceptual comparisons (picture first, then word, or word first, then picture). The symmetry of the visual and conceptual comparison results supports the hypothesis that the coding of the mind is neither intrinsically linguistic nor imagistic, but rather it is abstract. There is a potent stimulus size effect, equal for both pictorial and lexical stimuli. Small stimuli take longer to be visually processed than do larger stimuli. For optimal processing, stimuli should not only be equated for size, but should subtend a visual angle of at least 3 degrees. The article ends with the presentation of a mathematical theory that jointly accounts for the data from word-reading, picture-naming visual comparison, and conceptual-comparison experiments.     

Visual priming effects as a measure of short-term visual memory

On each trial, subjects classified one of four letters as belonging to one of two categories. Visual priming occurs when the classification response is faster to a stimulus visually identical to a previous stimulus than to one identical only in name. Earlier experiments found no visual priming effects between stimuli separated by a stimulus of the same task but from the opposite classification category. Two of the five conditions in the present experiment varied the stimulus-response (S-R) contingencies in such a way that the penultimate but not the immediately preceding trial had the same contingencies. Only these two conditions gave evidence of the above type of visual priming. Visual priming was found, however, in almost all conditions when the intervening stimulus was from the same task and the same classification category. It is argued that a similarity of S-R contingency, and not simply stimulus similarity, is an important component of the visual priming effect.     

The pseudohomophone effect: The role of visual similarity in non-word decisions

The pseudohomophone effect, that is, the finding that non-words that are pronounced like words (e.g. MEEN) take longer to reject in a lexical decision task than other pronounceable non-words (e.g. NEEN), has been used to support the hypothesis of phonological receding in lexical access. The lexical decision experiments reported here matched pseudohomophones to control non-words that were as visually similar to words as were the pseudohomophones. For both normal and aphasic subjects, reaction times to reject the pseudohomophones were no longer than those for the visual controls. However, the pseudohomophones did take longer to reject than other pronounceable non-words which were not as visually similar to words. The results suggest that most previous findings of the pseudohomophone effect result from the greater visual rather than phonological similarity of the pseudohomophones to words. The absence of a phonological effect on the non-words in the present study implies that phonological coding is an optional rather than a slow, but obligatory process.     

Semantic effects in visual word detection with visual similarity controlled

In three experiments, subjects searched through word lists for a single target word. Search was faster when the target word belonged to a different semantic category from that of the background words with visual similarity controlled. This semantic effect increased with number of items to be searched through, and it obtained whether the target was cued visually or verbally. Semantic homogeneity within the background also speeded search, but only when subjects had no prior knowledge of the nature of a list. Several models of the semantic effect are described. All contrast the encoding of physical identity with the encoding of semantic attributes, but they differ in ascribing the effect to: (1) the relative access times for these codes, (2) the power of the codes in dealing with multiple comparisons, or (3) the attentional demands of comparison using different levels of code.     

短时记忆中的范畴组织和通道组织的比较

本文通过设置不同的线索条件对短时记忆中范畴组织和通道组织进行了比较研究。实验采用了通道线索组、范围线索组和混合线索组三种条件,并设置了五种词表比例。在通道线索组中,每个词表是同范畴字词,但一半用视觉呈现,一半用听觉呈现;在范畴线索组中,两类词分属两个不同的词义范畴,但以相同通道呈现;在混合线索组中,两类不同范畴的词,各有一半分别用视觉和听觉呈现。结果表明在三种线索条件下,ＡＲＣ分值差异显著,表现为     

Semantic interference in a delayed naming task: evidence for the response exclusion hypothesis

In 2 experiments participants named pictures of common objects with superimposed distractor words. In one naming condition, the pictures and words were presented simultaneously on every trial, and participants produced the target response immediately. In the other naming condition, the presentation of the picture preceded the presentation of the distractor by 1,000 ms, and participants delayed production of their naming response until distractor word presentation. Within each naming condition, the distractor words were either semantic category coordinates of the target pictures or unrelated. Orthogonal to this manipulation of semantic relatedness, the frequency of the pictures' names was manipulated. The authors observed semantic interference effects in both the immediate and delayed naming conditions but a frequency effect only in the immediate naming condition. These data indicate that semantic interference can be observed when target picture naming latencies do not reflect the bottleneck at the level of lexical selection. In the context of other findings from the picture-word interference paradigm, the authors interpret these data as supporting the view that the semantic interference effect arises at a postlexical level of processing.     

Word length,phonemic, and visual similarity effects in poor and normal readers

Serial order recall for visually and auditorily presented stimuli was examined in a group of 12-year-old poor readers and 7-year-old reading-age controls. With pictorial presentation, the poor readers showed a visual similarity effect, no word length effect, and a smaller phonemic similarity effect than that of controls. However, with visual presentation of printed words and with auditory presentation, poor readers showed word length and phonemic similarity effects of similar magnitude to that of controls. It is concluded that poor readers rely on visual information in tasks where the presented images are highly codable, and where verbal recoding is not obligatory, but that they will make use of phonological coding when the stimuli are not as easily codable visually in memory.     

The effect of presentation format and encoding strategy on associative memory distortion

IntroductionPrevious studies using semantically related words revealed more accurate memory when the items were encoded visually rather than auditorily and when mental images were created during encoding. However, how the level of memory distortion is affected by the creation of different mental imagery formats or by techniques that should suppress generation of mental images has rarely been investigated.ObjectiveThe aim of the present studies was to investigate the ways in which the encoding strategy affects the accuracy of memory reports for two presentation formats of semantically related words: verbal and pictorial.MethodIn experiment 1, the participants were asked to memorize either pictures or their verbal equivalents (words) from the same category, using one of two encoding strategies: uttering the words or counting backwards. In experiment 2, pictorially or auditory presented material was encoded together with the creation of either visual or auditory mental images of the items. The results of the experimental groups were compared to control groups that received no specific instruction.ResultsHigher levels of false recognition, together with lower rates of correct recognition, were observed for words, presented either visually or auditory, relative to pictures. Moreover, self-generation of additional code during the processing of information favored the reduction of false recognitions.ConclusionEncoding strategies that engaged dual coding reduced false recognition. The results are discussed within the distinctiveness heuristic phenomenon.     

Spatial information with pictures and words in visual short-term memory

In a delayed matching task, the influence of spatial congruence between study and test on visual short-term memory for geometric figures and words was investigated. Subjects processed series of pictures which showed three words or three geometric figures arranged as rows or as triangular configurations. At test, the elements were presented in the identical or in the alternative configuration as at study. In the non-matching case, one of the studied elements was exchanged. The delay was 5 s. Subjects judged whether the elements were the same as during study, independent of their configuration. In Exp. 1, pictures of figures and words were mixed within one list. For both modalities, the response times were longer if the configuration at test was incongruent to the one at study. This contradicts the results of Santa, who observed effects of spatial congruency for figures, but not for words. In Exp. 2 we therefore presented the same material as in Exp. 1, but now the lists were modality-pure, as in the experiment of Santa – i.e., words and figures were shown in different lists. This time, spatial incongruency impaired recognition of the figures, but not recognition of the words. These results show that in a non-verbal context, isolated visually presented words are spatially encoded as non-verbal stimuli (figures) are. However, the word stimuli are encoded differently if the task is a pure verbal one. In the latter case, spatial information is discarded. Received: 9 September 1997 / Accepted: 30 March 1998     

Visual similarity effects in immediate verbal serial recall

The role of visual working memory in temporary serial retention of verbal information was examined in four experiments on immediate serial recall of words that varied in visual similarity and letters that varied in the visual consistency between upper and lower case. Experiments 1 and 2 involved words that were either visually similar (e.g. fly, cry, dry; hew, new, few ) or were visually distinct (e.g. guy, sigh, lie; who, blue, ewe ). Experiments 3 and 4 involved serial recall of both letter and case from sequences of letters chosen such that the upper- and lower-case versions were visually similar, for example Kk, Cc, Zz, Ww , or were visually dissimilar, for example Dd, Hh, Rr, Qq . Hence in the latter set, case informationwas encoded interms of both the shape and the size of the letters. With both words and letters, the visually similar items resulted in poorer recall both with and without concurrent articulatory suppression. This visual similarity effect was robust and was replicated across the four experiments. The effect was not restricted to any particular serial position and was particularly salient in the recall of letter case. These data suggest the presence of a visual code for retention of visually presented verbal sequences in addition to a phonological code, and they are consistent with the use of a visual temporary memory, or visual "cache", in verbal serial recall tasks.     

Lesioning an attractor network: investigations of acquired dyslexia

A recurrent connectionist network was trained to output semantic feature vectors when presented with letter strings. When damaged, the network exhibited characteristics that resembled several of the phenomena found in deep dyslexia and semantic-access dyslexia. Damaged networks sometimes settled to the semantic vectors for semantically similar but visually dissimilar words. With severe damage, a forced-choice decision between categories was possible even when the choice of the particular semantic vector within the category was not possible. The damaged networks typically exhibited many mixed visual and semantic errors in which the output corresponded to a word that was both visually and semantically similar. Surprisingly, damage near the output sometimes caused pure visual errors. Indeed, the characteristic error pattern of deep dyslexia occurred with damage to virtually any part of the network.     

Effects of bilingualism, aging, and semantic relatedness on memory under divided attention.

We examined how encoding and retrieval processes were affected by manipulations of attention, and whether the degree of semantic relatedness between words in the memory and distracting task modulated these effects. We also considered age and bilingual status as mediating factors. Monolingual and bilingual younger and older adults studied a list of words from a single semantic category presented auditorily, and later free recalled them aloud. During either study or retrieval, participants concurrently performed a distracting task requiring size decisions to words from either the same or a different semantic category as the words in the memory task. The greatest disruptions of memory from divided attention (DA) were for encoding rather than retrieval. The effect of semantic relatedness was significant only for DA at encoding. Older age and bilingualism were associated with lower recall scores in all conditions, but these factors did not influence the magnitude of memory interference. The results suggest that encoding is more sensitive to semantic similarity in a distracting task than is retrieval. The role of attention at encoding and retrieval is discussed.     

Integration of perceptual information in word access

According to experiential theories of language comprehension, perceptual information plays an essential role when word meanings are accessed. We conducted four experiments to investigate how different types of perceptual information such as colour and shape are combined during word access. One possibility is that the colour and shape of a word's referent are activated independently from one another and are combined in an additive manner. Alternatively, words might activate perceptual representations via a multiplicative integration of colour and shape. Experiment 1 established that participants follow a multiplicative similarity rule when they judge the similarity of schematic pictures to actual fruits and vegetables. In Experiments 2 to 4, participants performed a classification task, a lexical decision task, or a word-naming task on names of fruits and vegetables that were superimposed on a background picture. Responses were facilitated only when both colour and shape of the picture matched the word's referents. Response times were associated negatively with mean similarity ratings and the consistency of these ratings obtained in the first experiment. These results suggest a multiplicative integration of different types of perceptual information during word access.