An enquiry into the process of categorization of pictures and words

This paper reports a series of experiments conducted to study the categorization of pictures and words. Whereas some studies reported in the past have found a picture advantage in categorization, other studies have yielded no differences between pictures and words. This paper used an experimental paradigm designed to overcome some methodological problems to examine picture-word categorization. The results of one experiment were consistent with an advantage for pictures in categorization. To identify the source of the picture advantage in categorization, two more experiments were conducted. Findings suggest that semantic relatedness may play an important role in the categorization of both pictures and words. We explain these findings by suggesting that pictures simultaneously access both their concept and visually salient features whereas words may initially access their concept and may subsequently activate features. Therefore, pictures have an advantage in categorization by offering multiple routes to semantic processing.     

Semantic classification of pictures and words

We provide new behavioural norms for semantic classification of pictures and words. The picture stimuli are 288 black and white line drawings from the International Picture Naming Project ([Székely, A., Jacobsen, T., D'Amico, S., Devescovi, A., Andonova, E., Herron, D., et al. (2004). A new on-line resource for psycholinguistic studies. Journal of Memory & Language, 51, 247–250]). We presented these pictures for classification in a living/nonliving decision, and in a separate version of the task presented the corresponding word labels for classification. We analyzed behavioural responses to a subset of the stimuli in order to explore questions about semantic processing. We found multiple semantic richness effects for both picture and word classification. Further, while lexical-level factors were related to semantic classification of words, they were not related to semantic classification of pictures. We argue that these results are consistent with privileged semantic access for pictures, and point to ways in which these data could be used to address other questions about picture processing and semantic memory.     

Semantic facilitation with pictures and words

The present experiments explored the role of processing level and strategic factors in cross-form (word-picture and picture-word) and within-form (picture-picture and word-word) semantic facilitation. Previous studies have produced mixed results. The findings presented in this article indicate that semantic facilitation depends on the task and on the subjects' strategies. When the task required semantic processing of both picture and word targets (e.g., category verification), equivalent facilitation was obtained across all modality combinations. When the task required name processing (e.g., name verification, naming), facilitation was obtained for the picture targets. In contrast, with word targets, facilitation was obtained only when the situation emphasized semantic processing. The results are consistent with models that propose a common semantic representation for both picture and words but that also include assumptions regarding differential order of access to semantic and phonemic features for these stimulus modalities.     

Test expectancy and encoding of pictures and words

Undergraduates were shown pictures or corresponding labels and then were tested for recognition either in the same mode or in a cross-over mode. Significantly more items were recognized in the picture-picture condition than in the picture-word and word-picture conditions. Informing subjects in advance of the change in modality significantly improved picture-word performance.     

Selective disruption of hypermnesia for pictures and words

In this study, we investigated the effects of various interpolated tasks on hypermnesia (improved recall across repeated tests) for pictures and words. In five experiments, subjects studied either pictures or words and then completed two free-recall tests, with varying activities interpolated between the tests. The tasks performed between tests were varied to test several hypotheses concerning the possible factor(s) responsible for disruption of the hypermnesic effect. In each experiment, hypermnesia was obtained in a control condition in which there was no interpolated task between tests. The remaining conditions showed that the effect of the interpolated tasks was related to the overlap of the cognitive processes involved in encoding the target items and performing the interpolated tasks. When pictures were used as the target items, no hypermnesia was obtained when subjects engaged in imaginal processing interpolated tasks, even when these tasks involved materials that were very distinct from the target items. When words were used as the target items, no hypermnesia was obtained when the interpolated tasks required verbal/linguistic processing, even when the items used in these tasks were auditorily presented. The results are discussed in terms of a strength-based model of associative memory.     

Developmental differences in frequency judgments of words and pictures

Children in kindergarten, third, and fifth grades were presented a list of either pictures or words (with items presented for varying numbers of times on the study trial). In both picture and word conditions, half of the Ss estimated how many times each item had been presented (absolute judgments) and the other half judged which of two items had occurred more often on the study trial (relative judgments). The primary finding was that while frequency judgment performance improved with age for both pictures and words, there was relatively greater improvement for pictures (i.e., the picture-word difference increased with age). These results lend strong support to the frequency theory of discrimination learning and, in particular, may be useful in accounting for effects associated with age and with age by stimulus mode interactions.     

Situation models and retrieval interference: pictures and words

Previous studies have found that interference in long-term memory retrieval occurs when information cannot be integrated into a single situation model, but this interference is greatly reduced or absent when the information can be so integrated. The current study looked at the influence of presentation format-sentences or pictures-on this observed pattern. When sentences were used at memorisation and recognition, a spatial organisation was observed. In contrast, when pictures were used, a different pattern of results was observed. Specifically, there was an overall speed-up in response times, and consistent evidence of interference. Possible explanations for this difference were examined in a third experiment using pictures during learning, but sentences during recognition. The results from Experiment 3 were consistent with the organisation of information into situation models in long-term memory, even from pictures. This suggests that people do create situation models when learning pictures, but their recognition memory may be oriented around more "verbatim", surface-form memories of the pictures.     

Depression reduces perceptual sensitivity for positive words and pictures

There is evidence of maladaptive attentional biases for lexical information (e.g., Atchley, Ilardi, & Enloe, 2003; Atchley, Stringer, Mathias, Ilardi, & Minatrea, 2007) and for pictographic stimuli (e.g., Gotlib, Krasnoperova, Yue, & Joormann, 2004) among patients with depression. The current research looks for depressotypic processing biases among depressed out-patients and non-clinical controls, using both verbal and pictorial stimuli. A d′ measure (sensitivity index) was used to examine each participant's perceptual sensitivity threshold. Never-depressed controls evidenced a detection bias for positive picture stimuli, while depressed participants had no such bias. With verbal stimuli, depressed individuals showed specific decrements in the detection of positive person-referent words (WINNER), but not with positive non-person-referent words (SUNSHINE) or with negative words. Never-depressed participants showed no such differences across word types. In the current study, depression is characterised both by an absence of the normal positivistic biases seen in individuals without mood disorders (consistent with McCabe & Gotlib, 1995), and by a specific reduction in sensitivity for person-referent positive information that might be inconsistent with depressotypic self-schemas.     

Imagery ability and the identification of fragmented pictures and words

High and low imagery individuals completed two paper-and-pencil identification tasks involving fragmented pictures and fragmented words. Although a trend was present for spatial measures of imagery ability to correlate more highly with picture than with word identification, high imagers nevertheless surpassed low imagers on both tasks. An interpretation emphasizing the spatial-imaginal demands of both tasks is suggested. Significant correlations between word identification and some verbal measures, as well as self-reported imagery, may have resulted from the differential priming of verbal and imaginal strategies following subtle procedural changes in the two experiments described.     

Sex differences in memory estimates for pictures and words

Memory performance estimates of men and women before and after a recall test were investigated. College students (17 men and 20 women), all juniors, participated in a memory task involving the recall of 80 stimuli (40 pictures and 40 words). Before and after the task they were asked to provide estimates of their pre- and postrecall performance. Although no sex differences were found for total correct recall, recall for pictures, and recall for words, or in the estimates of memory performance before the recall task, there were significant differences after the test: women underestimated their performance on the words and men underestimated their performance on the picture items.     

Recalling pictures and words: Reversing the generation effect

Blind and sighted children’s memory for words and raised shape pictures was tested. The investigation compared performance with items when they were studied under neutral conditions (naming words and pictures) and when they were self-generated in response to cues (cue: hot ?: response: cold). The blind children could identify and name the raised shape pictures with the same apparent ease as blind-folded sighted children (as long as a cue was provided). The sighted children showed the generation effect (Slamecka and Graf, 1978) for both words and pictures, namely that self-generated items were far better recalled than neutral ones. The pattern of results for the blind children was markedly different. Although the level of memory performance overall was the same as that of the sighted controls, the congenitally blind children showed areverse generation effect. A stem completion study indicated that these results couldnot be accounted for by a relatively greater reliance on data-driven processing by the blind.     

Developmental changes in the semantic processing of pictures and words

Second- and fifth-graders' semantic decision times for pictures and words were analyzed relative to the predictions derived from unitary- and dual-memory models. At both grade levels, word-word response latencies were greater than picture-word latencies which, in turn, were greater than picture-picture latencies. An interaction between Grade and Condition indicated that verbal access times decreased more than pictorial access times. The data fit the predictions of a memory model postulating category storage in a single memory system as opposed to simultaneous representation in verbal and nonverbal memory systems. It was concluded that with increasing experience verbal access to this single semantic system is more rapid.     

Interactive visual imagery and memory search for words and pictures

In a short-term recognition memory experiment with words, subjects: (1) subvocally rehearsed the words, (2) generated a separate visual image for each word, (3) generated an interactive scene with such images, or (4) composed a covert sentence using the words in the memory set. Contrary to Seamon's (1972) results in a similar study, a serial memory search was found in all conditions, instead of the simultaneous scan which was expected when items were combined in interactive images. In a second study with pictures as stimuli, subjects who generated imaginal interactions between separate pictures, viewed interacting pictures, or viewed separate pictures also showed a serial search, i.e., longer RTs were obtained when more stimuli were held m memory. Since interactive imagery facilitated performance in an unexpected paired-associate task with memory set stimuli, one can argue that subjects actually processed or generated such interactions. It was suggested that memory search might not be simultaneous in tasks where the test stimulus constitutes only part of a memory image.