Integration of verbal and visual information as evidenced by distortions in picture memory.

Subjects were presented with a series of pictures, some of which were general (girl walking down the path) and others specific (girl hiking down the path). These pictures were matched with sentences which were either general or specific ("The girl is walking [hiking] down the path.") Subsequently, a forced-choice picture recognition test was administered in which subjects saw pairs of pictures and indicated which member of each pair they had seen before. It was found that labelling the picture with a sentence containing a specific verb substantially increased the likelihood that the specific picture corresponding to that verb would subsequently be falsely recognized. The results are discussed in terms of current theories of memorial representation.     

Dissociation of the neural systems for working memory maintenance of verbal and nonspatial visual information

Working memory for names and faces was investigated to ascertain whether verbal and nonspatial visual information is maintained in working memory by separate neural systems. The subjects performed a delayed match-to-sample task for famous or unfamous faces and names and a sensorimotor control task. Several occipital, temporal, parietal, and prefrontal areas were activated during all memory delays, in comparison with the control delays. Greater delay activity for unfamous faces than for names was obtained in the right fusiform gyrus, right inferior frontal gyrus (IFG), right IFG/ precentral gyrus, and right medial superior frontal gyrus, whereas greater delay activity for unfamous names than for faces was observed in the precuneus, left insula/postcentral gyrus, and left IFG/ precentral gyrus. There was no significant difference in the prefrontal activity in the comparison between famous faces and names. Greater delay activity for famous names than for faces was obtained in visual association and parietal areas. The results indicate that there is a functional dissociation based on information type within the neural system that is responsible for working memory maintenance of verbal and nonspatial visual information.     

Semantic memory and verbal working memory correlates of N400 to subordinate homographs

N400 is an event-related brain potential that indexes operations in semantic memory conceptual space, whether elicited by language or some other representation (e.g., drawings). Language models typically propose three stages: lexical access or orthographic- and phonological-level analysis; lexical selection or word-level meaning and associate activation; and lexical integration, sentence- and discourse-level operations. The exact stage that N400 reflects is unknown, although opinion favors lexical integration over lexical selection. Surprisingly, little research has assessed relationships between neuropsychological measures of semantic memory fund of information or verbal working memory capacity and N400. Subjects performed a homograph disambiguation comprehension task with minimal working memory load. Short sentences read: The noun was adjective/verb. The nouns were either homographs or unambiguous. The adjective/verb was disambiguating for the homograph, and congruent or incongruent for the unambiguous noun. The primary noun of interest was the subordinate homograph. Comprehension of the subordinate meaning should correlate with semantic memory stores, reflecting greater knowledge. If N400 primarily reflects lexical access operations, it should also correlate with measures of semantic knowledge. If N400 reflects lexical integration, it should correlate with measures of working memory capacity. Comprehension errors were associated with semantic memory stores, but not working memory capacity. N400 was related to working memory capacity, but not semantic knowledge, suggesting that N400 primarily reflects late-stage working memory operations. N400 to subordinate disambiguating words was larger with greater working memory capacity, and thus may index the absolute capacity of working memory rather than difficulty in contextual integration.     

Enduring visual memory despite forced verbal rehearsal.

The ability to decide rapidly that two visual stimuli are nominally the same when they are also visually the same (the Posner effect) was examined for stimuli of increasing visual complexity (Experiment 1) and when a greater variety of visual differences between the two stimuli was employed (Experiment 2). When the two stimuli each consisted of a pair of letters or when to single-letter stimuli sometimes differed in both case and style, the Posner effect occurred even though subjects made overt verbal rehearsals of the first stimulus. The results suggest that losses of the Posner effect found under simpler circumstances are not attributable to a switch from visual to exclusively verbal coding.     

Automatic guidance of visual attention from verbal working memory

Previous studies have shown that visual attention can be captured by stimuli matching the contents of working memory (WM). Here, the authors assessed the nature of the representation that mediates the guidance of visual attention from WM. Observers were presented with either verbal or visual primes (to hold in memory, Experiment 1; to verbalize, Experiment 2; or merely to attend, Experiment 3) and subsequently were required to search for a target among different distractors, each embedded within a colored shape. In half of the trials, an object in the search array matched the prime, but this object never contained the target. Despite this, search was impaired relative to a neutral baseline in which the prime and search displays did not match. An interesting finding is that verbal primes were effective in generating the effects, and verbalization of visual primes elicited similar effects to those elicited when primes were held in WM. However, the effects were absent when primes were only attended. The data suggest that there is automatic encoding into WM when items are verbalized and that verbal as well as visual WM can guide visual attention.     

Changes from visual to verbal memory organization as a function of age

The false recognition procedure was used to determine the relative dominance of visual and verbal memory organization at two grade levels. The results indicated that visual encoding was predominant for first-graders, but that both visual and verbal encoding occurred with fourth-graders.     

Insensitivity of visual short-term memory to irrelevant visual information

Several authors have hypothesized that visuo-spatial working memory is functionally analogous to verbal working memory. Irrelevant background speech impairs verbal short-term memory. We investigated whether irrelevant visual information has an analogous effect on visual short-term memory, using a dynamic visual noise (DVN) technique known to disrupt visual imagery (Quinn & McConnell, 1996b). Experiment 1 replicated the effect of DVN on pegword imagery. Experiments 2 and 3 showed no effect of DVN on recall of static matrix patterns, despite a significant effect of a concurrent spatial tapping task. Experiment 4 showed no effect of DVN on encoding or maintenance of arrays of matrix patterns, despite testing memory by a recognition procedure to encourage visual rather than spatial processing. Serial position curves showed a one-item recency effect typical of visual short-term memory. Experiment 5 showed no effect of DVN on short-term recognition of Chinese characters, despite effects of visual similarity and a concurrent colour memory task that confirmed visual processing of the characters. We conclude that irrelevant visual noise does not impair visual short-term memory. Visual working memory may not be functionally analogous to verbal working memory, and different cognitive processes may underlie visual short-term memory and visual imagery.     

The role of individual differences in visual\verbal information processing preferences in visual\verbal source monitoring

Researchers have long sought to understand individual differences in false memory susceptibility. One particularly interesting direction in this area looks at individual differences in visual versus verbal styles of information processing. Prior work linking individual differences in these cognitive styles with false memory susceptibility has been mixed, likely due to a lack of research that distinguishes between visual and verbal styles as independent factors. The current study addresses this issue by linking independent visual and verbal scores on the Style of Processing scale with performance on a Picture-Word source-monitoring task in which 82 participants indicated if various objects were previously presented to them as words or images. The results of this study show that higher visual style of processing preferences are associated with improved levels of visual source-monitoring ability as indexed by a reduced number of source-monitoring errors involving the misattribution of verbally presented items to a visual source.     

Cognitive integration of verbal and vocal information in spoken sentences

The problem of interpreting messages that contain both verbal and vocal information is similar to the problem of information integration in impression formation, clinical judgment, and decision-making. While the linear model has proven powerful in many experimental studies, the double bind theory and literature on incongruent communication suggest that configural effects should occur, particularly in the processing of inconsistent messages. Four models of the integration process were evaluated for short spoken sentences. Results confirmed the superiority of the linear model, but individual differences were observed. Evidence was found that inconsistent sentences are processed differently than consistent sentences, but the effects of inconsistency are not reflected in a multiplicative component. Comparison of this task with other information integration tasks is presented. Problems with methodologies for evaluating vocal information are presented, and implications of the results for the double bind theory are discussed.     

Eye movements and the integration of visual memory and visual perception

Because visual perception has temporal extent, temporally discontinuous input must be linked in memory. Recent research has suggested that this may be accomplished by integrating the active contents of visual short-term memory (VSTM) with subsequently perceived information. In the present experiments, we explored the relationship between VSTM consolidation and maintenance and eye movements, in order to discover how attention selects the information that is to be integrated. Specifically, we addressed whether stimuli needed to be overtly attended in order to be included in the memory representation or whether covert attention was sufficient. Results demonstrated that in static displays in which the to-be-integrated information was presented in the same spatial location, VSTM consolidation proceeded independently of the eyes, since subjects made few eye movements. In dynamic displays, however, in which the to-be-integrated information was presented in different spatial locations, eye movements were directly related to task performance. We conclude that these differences are related to different encoding strategies. In the static display case, VSTM was maintained in the same spatial location as that in which it was generated. This could apparently be accomplished with covert deployments of attention. In the dynamic case, however, VSTM was generated in a location that did not overlap with one of the to-be-integrated percepts. In order to "move" the memory trace, overt shifts of attention were required.     

Updating information in verbal working memory and executive functioning

50 older adults (M age=77.9 yr., SD=7.3; 35 women and 15 men) were tested using the updating working-memory task. They were also given the neuropsychological Wisconsin Card Sorting Test, assumed to evaluate executive functioning and the frontal cortex. A factor analysis with age, education, and verbal ability partialled out was computed on the updating task outcomes and resulted in a two-factor solution, indicating that this task requires two independent processes, interpreted as reflecting a storage component and an updating component. Partial correlations with age, education, and verbal ability partialled out indicated that Wisconsin Card Sorting Test measures were significantly associated with the factor supposed to reflect the updating process. Such results appeared consistent with the model of working memory with a central executive system involved in the updating process and related to the executive-frontal functioning, and a phonological loop system involved in the storage of verbal information and not linked to executive-frontal functions.     

Semantic and visual memory codes in learning disabled readers

Two experiments investigated whether learning disabled readers' impaired recall is due to multiple coding deficiencies. In Experiment 1, learning disabled and skilled readers viewed nonsense pictures without names or with either relevant or irrelevant names with respect to the distinctive characteristics of the picture. Both types of names improved recall of nondisabled readers, while learning disabled readers exhibited better recall for unnamed pictures. No significant difference in recall was found between name training (relevant, irrelevant) conditions within reading groups. In Experiment 2, both reading groups participated in recall training for complex visual forms labeled with unrelated words, hierarchically related words, or without labels. A subsequent reproduction transfer task showed a facilitation in performance in skilled readers due to labeling, with learning disabled readers exhibiting better reproduction for unnamed pictures. Measures of output organization (clustering) indicated that recall is related to the development of superordinate categories. The results suggest that learning disabled children's reading difficulties are due to an inability to activate a semantic representation that interconnects visual and verbal codes.     

Simultaneous motor and verbal processing of visual information in a modified Stroop test

Color patches could simultaneously be manually counted and conflicting color words named in the time required for one of the tasks alone. The independence of verbal and nonverbal information processing resembles “split-brain” behavior. Naming color patches while counting color words took more time than would have been required for doing the individual tasks successively, indicating response competition in the standard Stroop test. Form or location stimuli substituted for color words reduced interference when naming color patches.     

Maintenance of item and order information in verbal working memory

Although verbal recall of item and order information is well-researched in short-term memory paradigms, there is relatively little research concerning item and order recall from working memory. The following study examined whether manipulating the opportunity for attentional refreshing and articulatory rehearsal in a complex span task differently affected the recall of item- and order-specific information of the memoranda. Five experiments varied the opportunity for articulatory rehearsal and attentional refreshing in a complex span task, but the type of recall was manipulated between experiments (item and order, order only, and item only recall). The results showed that impairing attentional refreshing and articulatory rehearsal similarly affected recall regardless of whether the scoring procedure (Experiments 1 and 4) or recall requirements (Experiments 2, 3, and 5) reflected item- or order-specific recall. This implies that both mechanisms sustain the maintenance of item and order information, and suggests that the common cumulative functioning of these two mechanisms to maintain items could be at the root of order maintenance.     

The neural correlates of updating information in verbal working memory

The aim of the present study was to re-examine cerebral areas subserving the updating function of the central executive with a running span task requiring subjects to watch strings of consonants of unknown length and then to recall serially a specific number of recent items. In order to dissociate more precisely the updating process from the storage function, a four-item instead of a six-item memory load was used, contrary to our previous study (Salmon et al., 1996). In addition, a serial recall procedure was preferred to a recognition procedure in order to suppress the use of visuospatial strategies. The most significant increase of rCBF occurred in the left frontopolar cortex (Brodmann's area 10), spreading to the left middle frontal (Brodmann's area 46). Results suggest that frontopolar activation underlies an updating process in working memory.     

The development of verbal and visual working memory processes: a latent variable approach

Working memory (WM) processing in children has been studied with different approaches, focusing on either the organizational structure of WM processing during development (factor analytic) or the influence of different task conditions on WM processing (experimental). The current study combined both approaches, aiming to distinguish verbal and visual processing in order to investigate WM development. We investigated recall performance under different task conditions in a sample of 5- to 13-year-olds, applying latent class regression analysis. In this analysis, we examined latent classes (subgroups) within the sample that differed in terms of processing type. The interpretations of the latent classes were validated internally using characteristics of the latent classes and externally using recall performance of words and figures. The results showed that children of different developmental stages used the same type of processing under the same conditions. However, due to developmental differences, their overall performances differed, showing groups of children who were successful in verbal or visual processing and groups of children who were not. This study shows and discusses the importance of disentangling the influence of task conditions from the influence of WM development when interpreting recall performance in children.