Verbal recoding of visual stimuli impairs mentalimagetransformations

Two experiments were carried out to test the hypothesis that verbal recoding of visual stimuli in short-term memory influences long-term memory encoding and impairs subsequent mental image operations. Easy and difficult-to-name stimuli were used. When rotated 90 degrees counterclockwise, each stimulus revealed a new pattern consisting of two capital letters joined together. In both experiments, subjects first learned a short series of stimuli and were then asked to rotate mental images of the stimuli in order to detect the hidden letters. In Experiment 1, articulatory suppression was used to prevent subjects from subvocal rehearsal when learning the stimuli, whereas in Experiment 2, verbal labels were presented with each stimulus during learning to encourage a reliance on the verbal code. As predicted, performance in the imagery task was significantly improved by suppression when the stimuli were easy to name (Experiment 1) but was severely disrupted by labeling when the stimuli were difficult to name (Experiment 2). We concluded that verbal recoding of stimuli in short-term memory during learning disrupts the ability to generate veridical mental images from long-term memory.     

Flexibility of working memory encoding in a sentence–picture–sound verification task

Dual-process accounts of working memory have suggested distinct encoding processes for verbal and visual information in working memory, but encoding for nonspeech sounds (e.g., tones) is not well understood. This experiment modified the sentence–picture verification task to include nonspeech sounds with a complete factorial examination of all possible stimulus pairings. Participants studied simple stimuli–pictures, sentences, or sounds–and encoded the stimuli verbally, as visual images, or as auditory images. Participants then compared their encoded representations to verification stimuli–again pictures, sentences, or sounds–in a two-choice reaction time task. With some caveats, the encoding strategy appeared to be as important or more important than the external format of the initial stimulus in determining the speed of verification decisions. Findings suggested that: (1) auditory imagery may be distinct from verbal and visuospatial processing in working memory; (2) visual perception but not visual imagery may automatically activate concurrent verbal codes; and (3) the effects of hearing a sound may linger for some time despite recoding in working memory. We discuss the role of auditory imagery in dual-process theories of working memory.     

Specific conditions for a selective deficit in memory for order in children with dyslexia

Short-term memory (STM) models distinguish between item and order memorization. The present study aims to explore how item and order STM are affected by the nature of the stimuli, the sequential versus simultaneous mode of presentation, the visual versus auditory presentation modality, the possibility of verbal recoding. A total of 20 children with dyslexia were matched one-by-one with 20 typically reading children on sex, age (8–14 years), and grade. Computerized STM tasks were administered while manipulating type (item vs. order), stimuli (letters vs. colors), sequentiality, input and output modality, as well as the presence/absence of articulatory suppression and distractors. General Linear Model analyses were conducted on accuracy scores for item and order STM. Both item and order recall scores were lower for children with dyslexia. Although order STM in the visual input condition turned out to be more impaired than item STM in the dyslexic group, both item and order memory impairments become evident when verbal recoding is prevented through articulatory suppression. Moreover, dyslexic children, unlike typical readers, were not facilitated by the linguistic nature of the stimuli to be remembered. The present findings suggest that the often-reported selective impairment of serial memory in dyslexia is restricted to stimuli that are verbal in nature or can be verbally recoded, whereas both item and order memory impairments become evident when verbal recoding is prevented through articulatory suppression. The presence of distractors is particularly detrimental to STM in the dyslexic group. The sensitivity to distractors, suppression, and stimuli in STM is predictive of reading performance.     

The representation of nonstructural information in visual memory: Evidence from image combination

Two experiments investigated the differential representation of the figure and ground of a picture in visual short-term and long-term memory. It is known (Hitch, Brandimonte, & Walker, 1995) that subjects find it more difficult to combine mental images of two separately presented pictures in order to identify a novel form when the two pictures are incongruent in color (i.e., when a black-on-white line drawing has to be combined with a white-on-black drawing). In the present experiments, thefigures were depicted in solid form to allow color congruity to be varied independently for figure and ground. Results showed a clear impairment in image combination when the to-be-combined figures were incongruent in color (black-on-gray and white-on-gray) but not when theirgrounds were incongruently colored (gray-on-black and gray-on-white). In this way, image combination was seen to be supported by a representation of the object depicted in the picture rather than by a literal representation of the picture itself (i.e., a pictorial code). In line with previous findings, the same representation was seen to support image combination based on short-term memory (Experiment 1) and long-term memory (Experiment 2), provided that in the latter case verbal recoding was precluded. When verbal recoding was allowed, image combination based on long-term memory was insensitive to color congruity, implying the involvement of a more abstract structural representation.     

Abilities and strategies of blind and sighted subjects in visuo-spatial imagery

An active visuo-spatial memory task was used in order to determine the characteristics of mental imagery in subjects with and without visual experience. Subjects were instructed to generate a mental representation of verbally presented 2D patterns that were placed in a grid and to indicate how many pattern elements were in corresponding positions in the two halves of the grid according to a specific grid axis (vertical or horizontal). Unexpectedly, results showed a similar performance in early blind, late blind and sighted subjects. However, subjects' debriefing showed that the three groups used different strategies. The sighted and the late blind subjects took advantage of a visuo-spatial strategy. They generated a mental image of the matrix and they simplified this image to maintain only the relevant information in memory. In contrast, the early blind subjects encoded each pattern element by its location in a (X,Y) coordinate system without visual representation. This indicates that both early and late blind subjects are able to perform an active visuo-spatial imagery task as well as sighted subjects although they use different strategies.     

Verbal mediation and memory for novel figural designs: a dual interference study

To the extent that all types of visual stimuli can be verbalized to some degree, verbal mediation is intrinsic in so-called "visual" memory processing. This impurity complicates the interpretation of visual memory performance, particularly in certain neurologically impaired populations (e.g., aphasia). The purpose of this study was to investigate the relative contributions of verbal mediation to recognition memory for visual stimuli that vary with respect to their amenability to being verbalized. In Experiment 1, subjects attempted to verbally describe novel figural designs during presentation and then identify them in a subsequent recognition memory test. Verbalizing these designs facilitated memory. Stimuli that were found to be easiest or most difficult to verbalize at the group level were retained for the second study. In Experiment 2, subjects evidenced superior recognition memory for the relatively easy to verbalize items. This advantage was attenuated in subjects who performed a concurrent verbal interference task during encoding, but not in those who performed an analogous visual interference task. These findings provide evidence that impoverished verbal mediation disproportionately impedes memory for visual material that is relatively easy to verbalize. Implications for the clinical assessment of visual memory are discussed.     

Verbal overshadowing of visual memories: some things are better left unsaid

It is widely believed that verbal processing generally improves memory performance. However, in a series of six experiments, verbalizing the appearance of previously seen visual stimuli impaired subsequent recognition performance. In Experiment 1, subjects viewed a videotape including a salient individual. Later, some subjects described the individual's face. Subjects who verbalized the face performed less well on a subsequent recognition test than control subjects who did not engage in memory verbalization. The results of Experiment 2 replicated those of Experiment 1 and further clarified the effect of memory verbalization by demonstrating that visualization does not impair face recognition. In Experiments 3 and 4 we explored the hypothesis that memory verbalization impairs memory for stimuli that are difficult to put into words. In Experiment 3 memory impairment followed the verbalization of a different visual stimulus: color. In Experiment 4 marginal memory improvement followed the verbalization of a verbal stimulus: a brief spoken statement. In Experiments 5 and 6 the source of verbally induced memory impairment was explored. The results of Experiment 5 suggested that the impairment does not reflect a temporary verbal set, but rather indicates relatively long-lasting memory interference. Finally, Experiment 6 demonstrated that limiting subjects' time to make recognition decisions alleviates the impairment, suggesting that memory verbalization overshadows but does not eradicate the original visual memory. This collection of results is consistent with a recording interference hypothesis: verbalizing a visual memory may produce a verbally biased memory representation that can interfere with the application of the original visual memory.     

Cross-modal priming and explicit memory in patients with verbal production deficits

Implicit memory is often thought to reflect an influence of past experience on perceptual processes, yet priming effects are found when the perceptual format of stimuli changes between study and test episodes. Such cross-modal priming effects have been hypothesized to depend upon stimulus recoding processes whereby a stimulus presented in one modality is converted to other perceptual formats. The present research examined recoding accounts of cross-modal priming by testing patients with verbal production deficits that presumably impair the conversion of visual words into auditory/phonological forms. The patients showed normal priming in a visual stem completion task following visual study (Experiment 1), but showed impairments following auditory study in both implicit (Experiment 2) and explicit (Experiment 3) stem completion. The results are consistent with the hypothesis that verbal production processes contribute to the recoding of visual stimuli and support cross-modal priming. The results also indicate that shared processes contribute to both explicit memory and cross-modal implicit memory.     

An experimental study of visual imagination and memory

It is shown that an irrelevant visual perception interferes more with verbal learning by means of imagery than does an irrelevant auditory perception. The relative interfering effects of these perceptions were reversed in a verbal learning task involving highly abstract materials. Such results implicate the existence of a true visual component in imaginal mediation. A theoretical model is presented in which a visual system and a verbal-auditory system are distinguished. The visual system controls visual perception and visual imagination. The verbal-auditory system controls auditory perception, auditory imagination, internal verbal representation, and speech. Attention can be more easily divided between the two systems than within either one taken by itself. Furthermore, the visual and verbal-auditory systems are functionally linked by information recoding operations. The application of mnemonic imagery appears to involve a recoding of initially verbal information into visual form, and then the encoding of a primarily visual schema into memory. During recall, the schema is decoded as a visual image, and then recoded once again into the verbal-auditory system. Evidence for such transformations is provided not only by the interference data, but also by an analysis of recall-errors made by Ss using mnemonic imagery.     

The influence of location and visual features on visual object memory

In five experiments, we examined the influence of contextual objects’ location and visual features on visual memory. Participants’ visual memory was tested with a change detection task in which they had to judge whether the orientation (Experiments 1A, 1B, and 2) or color (Experiments 3A and 3B) of a target object was the same. Furthermore, contextual objects’ locations and visual features were manipulated in the test image. The results showed that change detection performance was better when contextual objects’ locations remained the same from study to test, demonstrating that the original spatial configuration is important for subsequent visual memory retrieval. The results further showed that changes to contextual objects’ orientation, but not color, reduced orientation change detection performance; and changes to contextual objects’ color, but not orientation, impaired color change detection performance. Therefore, contextual objects’ visual features are capable of affecting visual memory. However, selective attention plays an influential role in modulating such effects.     

Memory while shadowing

Subjects were asked to repeat verbally (shadow) English words which were presented to one ear. They were then tested for their memory of two-digit numbers which were presented to their other ear. It is demonstrated that subjects have no memory for these digits if they must continue to shadow for 20 sec. before being tested on their memory for the digits. However, if tested immediately after the digit presentation, they do remember some digits. Hence, verbal material presented on non-attended channels gets into a short-term memory, but is not transferred into long-term memory.     

The role of working memory components and visuospatial abilities in route learning within a virtual environment

An experiment was run to complete our understanding of the involvement of working memory (WM) components in the construction of a spatial model from visual input, considering some of the visuospatial abilities known to modulate performance. In addition, to allow for consideration of the flexibility of the spatial representation, routes in a virtual environment were presented with a route perspective, and tests were presented with route and survey perspectives. The results indicate that the verbal and spatial WM are only involved in the memorisation of certain types of information, and that their involvement depends on the change of perspective necessitated by the task. Thus, even when the learning material and the tests used to assess performance are only visual, a verbal recoding of some information is necessary. Moreover, individual differences modulate the involvement of WM; an individual with higher visuospatial capacities uses more spatial WM than an individual with lower spatial capacities.     

Word length,phonemic, and visual similarity effects in poor and normal readers

Serial order recall for visually and auditorily presented stimuli was examined in a group of 12-year-old poor readers and 7-year-old reading-age controls. With pictorial presentation, the poor readers showed a visual similarity effect, no word length effect, and a smaller phonemic similarity effect than that of controls. However, with visual presentation of printed words and with auditory presentation, poor readers showed word length and phonemic similarity effects of similar magnitude to that of controls. It is concluded that poor readers rely on visual information in tasks where the presented images are highly codable, and where verbal recoding is not obligatory, but that they will make use of phonological coding when the stimuli are not as easily codable visually in memory.     

Testing visual short-term memory: Simultaneous versus sequential presentations

Two experiments testing immediate ordered recall are presented; in these experiments, subjects engaged in repetitive speech (“articulatory suppression”) during a visual presentation in order to prevent auditory recoding of the stimuli. In both experiments, a simultaneous presentation produced results that suggested the use of visual short-term memory, whereas a sequential presentation did not. In Experiment 1, visual confusion errors occurred more often than would be expected by chance for a simultaneous presentation but not for a sequential presentation. In Experiment 2, recall from visual short-term memory was expected to suffer more when subjects wrote a prefix than when they spoke a prefix; this effect occurred for a simultaneous presentation but not for a sequential presentation. These results suggest that existence of a visual short term store that retains a simultaneous presentation but not a sequential presentation.     

The Representation of Left-Right Orientation: A Dissociation Between Imagery and Perceptual Recognition

Eight experiments were conducted to determine whether visual mental imagery preserves left-right orientation in conditions where recognition memory apparently does not. Such a dissociation would suggest that information in memory preserves left-right orientation, but the process that matches input to such representations does not respect this distinction. In the experiments reported here, participants were asked to use one of two methods to indicate which way Abraham Lincoln faces on a US penny: Either they selected the correct profile from two sketches, or they formed a mental image and then specified the direction. If participants had just viewed pennies and were asked to visualize a particular one, they performed far better than chance; in contrast, if imagery was discouraged, they performed poorly when recognizing the profiles perceptually. However, unless actively discouraged from doing so, most participants reported spontaneously visualizing pennies prior to perceptual recognition if they had recently seen pennies.     

Simple movement imitation: Are kinematic features sufficient to map perceptions into actions?

The aim of this study was to pinpoint the nature of the visual features used in the automatic mapping of perceived movements into similar executed movements, following the direct matching hypothesis. In Experiment 1 subjects imitated the lifting of one of two fingers, presented with different orientations. As predicted, stimuli which were further rotated away from the posture of the executing hand elicited slower reaction times. In Experiment 2, we verified that this orientation effect was not a purely perceptual effect by presenting the same stimuli but asking subjects to respond verbally. No orientation effect was found using a verbal response. In Experiment 3, we replaced the moving fingers by two arbitrary objects moving with the trajectories of the finger tips of Experiment 1. The same orientation effect as in Experiment 1 was observed. We conclude that in this experiment participants are using purely kinematic features to map perceived into executed movements.     

没听到？“听”到！——来自听觉表象神经机制研究的证据

近年来听觉表象开始得到关注,相关研究包括言语声音、音乐声音、环境声音的听觉表象三类。本文梳理了认知神经科学领域对上述三种听觉表象所激活的脑区研究,比较了听觉表象和听觉对应脑区的异同,并展望了听觉表象未来的研究方向。