A comparison of conscious and automatic memory processes for picture and word stimuli: a process dissociation analysis

Four experiments were conducted to evaluate explanations of picture superiority effects previously found for several tasks. In a process dissociation procedure (Jacoby, 1991) with word stem completion, picture fragment completion, and category production tasks, conscious and automatic memory processes were compared for studied pictures and words with an independent retrieval model and a generate-source model. The predictions of a transfer appropriate processing account of picture superiority were tested and validated in "process pure" latent measures of conscious and unconscious, or automatic and source, memory processes. Results from both model fits verified that pictures had a conceptual (conscious/source) processing advantage over words for all tasks. The effects of perceptual (automatic/word generation) compatibility depended on task type, with pictorial tasks favoring pictures and linguistic tasks favoring words. Results show support for an explanation of the picture superiority effect that involves an interaction of encoding and retrieval processes.     

How are pictures registered in memory?

Three experiments tested the hypothesis that pictorial memory is much less dependent on rehearsal than is verbal memory. Experiment I examined incidental learning since this is assumed to reflect learning with little or no rehearsal. Following a classification task, intentional and incidental learning for pictures and for words was compared. The superiority of pictorial memory was especially marked in incidental learning. Experiment II showed that this result was not due to differences in the amount of processing required to classify pictures and words. RTs to classify words and pictures did not differ, and incidental learning was again superior for pictures. In Experiment III rehearsal opportunity was restricted by a concurrent task during presentation of word and picture lists, and the decrement was very much greater for word learning than for picture learning. It was concluded that manipulation of rehearsal opportunity has relatively little effect on pictorial memory.     

Right-hemisphere interactions in picture-word processing

The processing of pictures was investigated in three experiments which eliminated the response effects involved in naming. When a categorization task was used, clear advantages in response latency and accuracy were observed for left visual-field (LVF) presentations. This was in contrast to previous investigations which have used a naming task and which have reported right visual-field (RVF) advantages. When a distracting word was added to the display, the pattern of influence also changed from that reported previously. The use of naming tasks has indicated predominantly left-hemisphere effects, with demonstrations of interactions between pictures and words in the RVF. With a categorization task in Experiment 2, however, the only effective words were those related in meaning to the picture, and only when they were projected to the right hemisphere. The third experiment confirmed the LVF advantage for picture processing with masked displays, but found no reliable asymmetry with unmasked presentations. The pattern of semantic facilitation was also confirmed with the masked displays, but when the mask was removed an inhibition effect replaced the facilitation effect. These effects are interpreted as indicating that picture recognition is localized within the right cerebral hemisphere. It is suggested that the facilitating effect of related words is restricted to the left hemisphere because it is an effect upon recognition processes, whereas the inhibition effect reflects response competition. It is also suggested that previous reports of left-hemisphere interference effects are due to effects of response competition in naming tasks.     

Task-hemispheric integrity in dual task performance

A condition of task hemispheric integrity is predicted to result in a dual task situation when the central processing and response components of each task are associated exclusively with a given cerebral hemisphere. The prediction that this condition will generate more efficient time-sharing is tested in a series of four experiments. In experiment 1 the prediction is confirmed when a spatial (tracking) and verbal (letter memory search) task are time-shared, and the hand assignment to task responses is manipulated. In experiment 2 a spatial variant of the memory search is used instead of the verbal letter search, and hand assignment effects are not obtained, since when two spatial tasks are time-shared an integrity assignment is impossible. Experiment 3 validates the hemispheric lateralization of the two single task variants of the memory search task, while experiment 4 establishes the separate “spatial’ and “verbal” resource demands of the two variants by observing their differential interference with concurrent spatial and verbal tasks.     

Hemispheric asymmetry,early visual processes,and serial memory comparison

Observers indicated whether a single probe letter presented to the left visual field/right hemisphere (LVF-RH) or to the right visual field/left hemisphere (RVF-LH) matched one of two, three, four, or five set letters in both name and case. For positive trials during the initial experimental session, the slope of the linear memory set size reaction time function was increased by perceptually degrading the probe letter on LVF-RH trials, but not on RVF-LH trials. In addition, perceptual degradation of the probe letter increased the intercept of the memory set size function more on RVF-LH trials than on LVF-RH trials. During subsequent experimental sessions, the same pattern of intercept results was obtained but perceptual degradation of the probe no longer changed the slope for either LVF-RH or RVF-LH trials. However, the slopes were uniformly lower on RVF-LH trials than on LVF-RH trials. The major results are consistent with hypothesized right-hemisphere efficiency for early processing stages and left-hemisphere efficiency for serial processing of alphanumeric information. The results further illustrate the importance of separating stages of processing in studies of cerebral laterality and indicate that the relative difficulty of the various stages can be a determinant of laterality results.     

Hemispheric patterns in visual search

A visual search paradigm was employed to examine hemispheric serial and parallel processing. Stimulus arrays containing 4, 9, or 16 elements were tachistoscopically presented to the right visual field-left hemisphere (RVF-LH) or left visual field-right hemisphere (LVF-RH). Subjects judged whether all of the elements within an array were physically the same (all X's) or whether one (O) was different from the rest. Left hemisphere presentations were processed more quickly and accurately than LVF-RH presentations for all stimulus conditions. As the number of array elements increased, more errors and longer response times were obtained for different stimulus items whereas fewer errors and somewhat shorter response times were obtained for same stimulus items. These and previous results suggest that the left hemisphere obtains an advantage for visual search because of that hemisphere's superiority for fine-grained feature analysis rather than because of a fundamental hemispheric serial/parallel processing dichotomy.     

Effects of perceptual quality on the processing of human faces presented to the left and right cerebral hemispheres

Three experiments examined the effects of stimulus duration, retinal eccentricity, and visual noise on the processing of human faces presented to the left visual field/right hemisphere (LVF-RH) and right visual field/left hemisphere (RVF-LH). In Experiment 1 observers identified which of 10 similar male faces was presented on a screen. The single face was presented for 10, 55, or 100 ms at 1 degree, 4 degrees, or 9 degrees of visual angle to the left or right of fixation. Decreasing stimulus duration and increasing retinal eccentricity lowered face recognition. The effect of duration was the same for LVF-RH and RVF-LH trials, but the detrimental effect of increasing retinal eccentricity was larger on LVF-RH trials than on RVF-LH trials. In Experiment 2 observers indicated whether a single face from this same set was a member of a memorized set of five positive faces. The probe face on each trial was presented alone or embedded in visual noise. Visual noise increased the error rate more on LVF-RH trials than on RVF-LH trials. This effect was replicated in Experiment 3, which also required observers to make a much easier discrimination between male and female faces. In the male/female task visual noise tended to impair performance more on RVF-LH trials than on LVF-RH trials, opposite the effect for the male/male task. These results are discussed in terms of hemispheric asymmetry for global versus local features of faces, the level of feature analysis demanded by a task, and the level of feature analysis most disrupted by perceptual degradation.     

Hemispheric differences in picture-word interference

In a picture-word version of the Stroop task, 30 right-handed subjects were tested under each of six conditions in which a picture alone or a picture plus a word were presented to the left, the right, or both hemispheres. In two additional conditions the picture was presented to the right hemisphere and the word was simultaneously presented to the left hemisphere, or vice versa. For all conditions, subjects were instructed to name the picture only, as rapidly as possible. Picture naming times were significantly slower for the conditions in which the pictures were accompanied by words than in the respective picture alone conditions. Moreover, simultaneous presentation of a picture and a word to both hemispheres resulted in greater interference (slower picture naming times) than did the simultaneous presentation of the picture and the word to either the left hemisphere alone or the right hemisphere alone. The latter two conditions, in turn, resulted in significantly more interference than did the simultaneous presentation of the picture to one hemisphere and the word to the other hemisphere. This pattern of results suggests that the Stroop effect obtained under normal circumstances is in large part a function of the interference caused by the simultaneous processing of items in the same hemisphere. In contrast to hemispheric differences reported for the color-word Stroop task, the effect of presenting a picture and word simultaneously to the right hemisphere did not differ reliably from that of presenting a picture and word to the left hemisphere. The failure to replicate this aspect of the color-word Stroop is attributed to differences in the abilities of the two hemispheres to process the respective target items (the color or the picture) of the two tasks.     

Learning 10000 pictures

Four experiments are reported which examined memory capacity and retrieval speed for pictures and for words. Single-trial learning tasks were employed throughout, with memory performance assessed by forced-choice recognition, recall measures or choice reaction-time tasks. The main experimental findings were: (1) memory capacity, as a function of the amount of material presented, follows a general power law with a characteristic exponent for each task; (2) pictorial material obeys this power law and shows an overall superiority to verbal material. The capacity of recognition memory for pictures is almost limitless, when measured under appropriate conditions; (3) when the recognition task is made harder by using more alternatives, memory capacity stays constant and the superiority of pictures is maintained; (4) picture memory also exceeds verbal memory in terms of verbal recall; comparable recognition/recall ratios are obtained for pictures, words and nonsense syllables; (5) verbal memory shows a higher retrieval speed than picture memory, as inferred from reaction-time measures. Both types of material obey a power law, when reaction-time is measured for various sizes of learning set, and both show very rapid rates of memory search.

From a consideration of the experimental results and other data it is concluded that the superiority of the pictorial mode in recognition and free recall learning tasks is well established and cannot be attributed to methodological artifact.     

Automatic semantic association between emotional valence and brightness in the right hemisphere

Positive words (e.g., faith) were recognised better when presented in white fonts than in black fonts, whereas the opposite was true for negative words (e.g., enemy). A neural basis for this type of association between emotional valence and brightness was investigated using a visual half-field paradigm. Positive and negative words were presented in black or white fonts and presented to the left visual field-right hemisphere (LVF-RH) or right visual field-left hemisphere (RVF-LH) in a word valence judgement task (i.e., positive vs. negative). A cross-over interaction between emotional valence and brightness was observed; valence judgements were facilitated when a positive word appeared in white and when a negative word appeared in black. This interaction was qualified by a higher-order interaction. The cross-over interaction appeared only for LVF-RH trials, suggesting that the right hemisphere was responsible for the association between emotional valence and brightness.     

Picture superiority in conceptual memory: Dissociative effects of encoding and retrieval tasks

We examined the role of encoding processes for picture superiority in explicit and conceptual-implicit memory. The nature of encoding instruction (naming or semantic categorization) yielded dissociative effects on picture and word memory on one explicit test, category-cued recall, and two conceptual-implicit tests, category-cued generation and category-cued verification. Category-cued recall was greater for pictures than for words following naming, but it did not differ for pictures and words following semantic categorization. Category-cued generation priming was greater for pictures than for words following naming, but it was greater for words than for pictures following semantic categorization. In contrast, category-cued verification priming did not differ for pictures and words following either naming or semantic categorization. Thus, picture superiority can be eliminated or reversed depending on the type of conceptual encoding task and conceptual-retrieval test.     

Hemispheric specialization for categorical and coordinate spatial representations: A reappraisal

The purpose of the present study was to examine Kosslyn's (1987) claim that the left hemisphere (LH) is specialized for the computation of categorical spatial representations and that the right hemisphere (RH) is specialized for the computation of coordinate spatial representations. Categorical representations involve making judgements about the relative position of the components of a visual stimulus (e.g., whether one component is above/below another). Coordinate representations involve calibrating absolute distances between the components of a visual stimulus (e.g., whether one component is within 5 mm of another). Thirty-two male and 32 female undergraduates were administered two versions of a categorical or a coordinate task over three blocks of 36 trials. Within each block, items were presented to the right visual field-left hemisphere (RVF-LH), the left visual field-right hemisphere (LVF-RH), or a centralized position. Overall, results were more supportive of Kosslyn's assertions concerning the role played by the RH in the computation of spatial representations. Specifically, subjects displayed an LVF-RH advantage when performing both versions of the coordinate task. The LVF-RH advantage on the coordinate task, however, was confirmed to the first block of trials. Finally, it was found that males were more likely than females to display faster reaction times (RTs) on coordinate tasks, slower RTs on categorical tasks, and an LVF-RH advantage in computing coordinate tasks.     

Semantic component of a cross-modal Stroop-like task

Three experiments showed that the pattern of interference of single-modality Stroop tests also exists cross-modally. Distractors and targets were either pictures or auditory words. In a naming task (Experiment 1), word distractors from the same semantic category as picture targets interfered with picture naming more than did semantically unrelated distractors; the semantic category of picture distractors did not differentially affect word naming. In a categorization task (Experiment 2), this Stroop-like effect was reversed: Picture distractors from the same semantic category as word targets interfered less with word categorization than picture distractors that were semantically unrelated; the semantic category of word distractors did not differentially affect picture categorization. Experiment 3 replicated these effects when each subject performed both tasks; the task, naming or categorizing, determined the pattern of interference between pictures and auditory words. The results thus support the existence of a semantic component of a cross-modal Stroop-like effect.     

Hemispheric asymmetries for temporal resolution: A signal detection analysis of threshold and bias

Divided visual field techniques were used to investigate hemispheric asymmetries for (a) the threshold of fusion of two flashes of light and (b) the detection of simultaneous versus successive events for a group of normal, right-handed adults. A signal detection analysis revealed a higher level of accuracy for the right visual field-left hemisphere (RVF-LH) relative to the left visual field-right hemisphere (LVF-RH) for both tasks. These results were interpreted in terms of a general left-hemisphere advantage for the discrimination of fine temporal events. The implications of these results for models of temporary asymmetry that describe the left hemisphere's advantage in terms of an exclusive specialization or relative superiority are then discussed.     

Valence specific laterality effects in free viewing conditions: the role of expectancy and gender of image

Recent research has looked at whether the expectancy of an emotion can account for subsequent valence specific laterality effects of prosodic emotion, though no research has examined this effect for facial emotion. In the study here (n=58), we investigated this issue using two tasks; an emotional face perception task and a novel word task that involved categorising positive and negative words. In the face perception task a valence specific laterality effect was found for surprise (positive) and anger (negative) faces in the control but not expectancy condition. Interestingly, lateralisation differed for face gender, revealing a left hemisphere advantage for male faces and a right hemisphere advantage for female faces. In the word task, an affective priming effect was found, with higher accuracy when valence of picture prime and word target were congruent. Target words were also responded to faster when presented to the LVF versus RVF in the expectancy but not control condition. These findings suggest that expecting an emotion influences laterality processing but that this differs in terms of the perceptual/experience dimension of the task. Further, that hemispheric processing of emotional expressions appear to differ in the gender of the image.     

Effects of perceptual quality and visual field of probe stimulus presentation on memory search for letters

Observers indicated whether a single probe letter presented to the left visual field/right hemisphere (LVF-RH) or to the right visual field/left hemisphere (RVF-LH) was contained in a memory set of 2, 3, 4, or 5 letters. For positive trials, the increase in reaction time caused by perceptually degrading the probe letter became progressively larger as memory set size became larger when the probe was presented to the LVF-RH but not when the probe was presented to the RVF-LH. These results were obtained regardless of whether the case of the probe letter varied randomly (Experiment 1) or only capital letters were used (Experiment 2). The results on LVF-RH trials suggest a relatively visuospatial memory comparison process, whereas the results on RVF-LH trials suggest a more abstract memory comparison process. In addition to these effects, the intercept of the memory set size function was lower on LVF-RH trials than on RVF-LH trials when the probe letter was perceptually degraded, consistent with the hypothesis that the right hemisphere is more efficient than the left at early visuospatial processes. Perhaps it is this efficiency at early visuospatial processes that produces the bias toward visuospatial memory comparison on LVF-RH trials.     

Episodic encoding and recognition of pictures and words: role of the human medial temporal lobes

In the present PET study, we examined brain activity related to processing of pictures and printed words in episodic memory. Our goal was to determine how the perceptual format of objects (verbal versus pictorial) is reflected in the neural organization of episodic memory for common objects. We investigated this issue in relation to encoding and recognition with a particular focus on medial temporal-lobe (MTL) structures. At encoding, participants saw pictures of objects or their written names and were asked to make semantic judgments. At recognition, participants made yes-no recognition judgments in four different conditions. In two conditions, target items were pictures of objects; these objects had originally been encoded either in picture or in word format. In two other conditions, target items were words; they also denoted objects originally encoded either as pictures or as words. Our data show that right MTL structures are differentially involved in picture processing during encoding and recognition. A posterior MTL region showed higher activation in response to the presentation of pictures than of words across all conditions. During encoding, this region may be involved in setting up a representation of the perceptual information that comprises the picture. At recognition, it may play a role in guiding retrieval processes based on the perceptual input, i.e. the retrieval cue. Another more anterior right MTL region was found to be differentially involved in recognition of objects that had been encoded as pictures, irrespective of whether the retrieval cue provided was pictorial or verbal in nature; this region may be involved in accessing stored pictorial representations. Our results suggest that left MTL structures contribute to picture processing only during encoding. Some regions in the left MTL showed an involvement in semantic encoding that was picture specific; others showed a task-specific involvement across pictures and words. Together, our results provide evidence that the involvement of some but not all MTL regions in episodic encoding and recognition is format specific.     

Tracing the time course of picture--word processing

A number of independent lines of research have suggested that semantic and articulatory information become available differentially from pictures and words. The first of the experiments reported here sought to clarify the time course by which information about pictures and words becomes available by considering the pattern of interference generated when incongruent pictures and words are presented simultaneously in a Stroop-like situation. Previous investigators report that picture naming is easily disrupted by the presence of a distracting word but that word naming is relatively immune to interference from an incongruent picture. Under the assumption that information available from a completed process may disrupt an ongoing process, these results suggest that words access articulatory information more rapidly than do pictures. Experiment 1 extended this paradigm by requiring subjects to verify the category of the target stimulus. In accordance with the hypothesis that picture access the semantic code more rapidly than words, there was a reversal in the interference pattern: Word categorization suffered considerable disruption, whereas picture categorization was minimally affected by the presence of an incongruent word. Experiment 2 sought to further test the hypothesis that access to semantic and articulatory codes is different for pictures and words by examining memory for those items following naming or categorization. Categorized words were better recognized than named words, whereas the reverse was true for pictures, a result which suggests that picture naming involves more extensive processing than picture categorization. Experiment 3 replicated this result under conditions in which viewing time was held constant. The last experiment extended the investigation of memory differences to a situation in which subjects were required to generate the superordinate category name. Here, memory for categorized pictures was as good as memory for named pictures. Category generation also influenced memory for words, memory performance being superior to that following a yes--no verification of category membership. These experiments suggest a model of information access whereby pictures access semantic information were readily than name information, with the reverse being true for words. Memory for both pictures and words was a function of the amount of processing required to access a particular type of information as well as the extent of response differentiation necessitated by the task.     

Valence specific laterality effects in free viewing conditions: The role of expectancy and gender of image

Recent research has looked at whether the expectancy of an emotion can account for subsequent valence specific laterality effects of prosodic emotion, though no research has examined this effect for facial emotion. In the study here (n = 58), we investigated this issue using two tasks; an emotional face perception task and a novel word task that involved categorising positive and negative words. In the face perception task a valence specific laterality effect was found for surprise (positive) and anger (negative) faces in the control but not expectancy condition. Interestingly, lateralisation differed for face gender, revealing a left hemisphere advantage for male faces and a right hemisphere advantage for female faces. In the word task, an affective priming effect was found, with higher accuracy when valence of picture prime and word target were congruent. Target words were also responded to faster when presented to the LVF versus RVF in the expectancy but not control condition.These findings suggest that expecting an emotion influences laterality processing but that this differs in terms of the perceptual/experience dimension of the task. Further, that hemispheric processing of emotional expressions appear to differ in the gender of the image.     

Cerebral lateralization of language in deaf and hearing people

In Experiment 1 neither hearing nor prelingually deaf signing adolescents showed marked lateralization for lexical decision but, unlike the hearing, the deaf were not impaired by the introduction of pseudohomophones. In Experiment 2 semantic categorization produced a left hemisphere advantage in the hearing for words but not pictures whereas in the deaf words and signs but not pictures showed a right hemisphere advantage. In Experiment 3 the lexical decision and semantic categorization findings were confirmed and both groups showed a right hemisphere advantage for a face/nonface decision task. The possible effect of initial language acquisition on the development of hemispheric lateralization for language is discussed.