A review of processing and analysing visual metaphors in psychology

IntroductionA visual metaphor is an image depicting an incongruity in the spatial distribution of visual elements, which is not consistent with reality. In this paper, we present an overview of theoretical and empirical studies on visual metaphor processing.ObjectiveFirstly, attention is given to the spatial distribution of visual elements, which defines the type of visual metaphors as well as the meaning operations required to understand the communicative message. The paper then reviews several empirical studies that collect behavioural measures for assessing visual metaphor processing using questionnaires and exploring the role played by cognitive abilities. In line with the contemporary literature, we then present three semiotic dimensions for visual metaphor processing, namely the expression, conceptualisation, and communication. We then present a pilot study that focusses on these three dimensions. Few research studies have collected behavioural data for visual metaphors processing. This may be due to the lack of theoretical framework in the corresponding field of research. These three semiotic dimensions highlight the value of pursuing research on visual metaphors in the scope of psychology. The presented pilot study could be a starting point for the investigation of visual metaphors in the framework of psychology.ConclusionWe propose future avenues of research that consider the visual structure of metaphors and meaning operations while assessing the influence of cognitive abilities in visual metaphor processing.     

Automatic and attentional processes in the comprehension of unfamiliar metaphors

The present study employed priming techniques to investigate whether unfamiliar metaphors are understood automatically, or whether controlled processes are required for comprehension. Priming sentences were presented to 76 participants in a naming task and vocal RT (reaction time) to target words was recorded. Target words were associates of the 120 metaphors used in the stimulus set. SOA (stimulus onset asynchrony) was manipulated in order to permit the measurement of both automatic and attentional priming (375ms, 750ms, and 1,500ms). Four priming conditions enabled a comparison between metaphoric, literal, and neutral contexts, and an unrelated (baseline) condition. The data revealed a strong priming effect for metaphors at an SOA of 1,500ms, but no effects at either of the shorter SOAs. These results indicate that, in the absence of supporting context, unfamiliar metaphors are not processed automatically. Comprehension of such linguistic phenomena apparently requires attentional processing. The findings have implications for direct processing theories of metaphor comprehension.     

Automatic and attentional processes in the comprehension of unfamiliar metaphors

The present study employed priming techniques to investigate whether unfamiliar metaphors are understood automatically, or whether controlled processes are required for comprehension. Priming sentences were presented to 76 participants in a naming task and vocal RT (reaction time) to target words was recorded. Target words were associates of the 120 metaphors used in the stimulus set. SOA (stimulus onset asynchrony) was manipulated in order to permit the measurement of both automatic and attentional priming (375ms, 750ms, and 1,500ms). Four priming conditions enabled a comparison between metaphoric, literal, and neutral contexts, and an unrelated (baseline) condition. The data revealed a strong priming effect for metaphors at an SOA of 1,500ms, but no effects at either of the shorter SOAs. These results indicate that, in the absence of supporting context, unfamiliar metaphors are not processed automatically. Comprehension of such linguistic phenomena apparently requires attentional processing. The findings have implications for direct processing theories of metaphor comprehension.     

On memory for metaphor

Three experiments investigated how metaphors are represented in memory and the effects on memory of variables known to affect metaphor comprehension. Ten theoretically relevant dimensions were examined. In Experiments 1 and 2, free recall across a variety of orienting tasks was consistently and positively predicted by the rated imageability of metaphoric topics (sentence subjects) and the rated number of interpretations for each metaphor. The number of interpretations effect was reversed in the cued recall task of Experiment 3, as metaphors with fewer interpretations were remembered better. These and other results supported previous suggestions that memory for metaphor involves wholistic representations akin to metaphoric grounds. They also revealed consistent differences in the roles of several variables in comprehension and memory for metaphor.     

Kill the Song—Steal the Show: What Does Distinguish Predicative Metaphors From Decomposable Idioms?

This study examined the semantic processing difference between decomposable idioms and novel predicative metaphors. It was hypothesized that idiom comprehension results from the retrieval of a figurative meaning stored in memory, that metaphor comprehension requires a sense creation process and that this process difference affects the processing time of idiomatic and metaphoric expressions. In the first experiment, participants read sentences containing decomposable idioms, predicative metaphors or control expressions and performed a lexical decision task on figurative targets presented 0, 350, and 500 ms, or 750 after reading. Results demonstrated that idiomatic expressions were processed sooner than metaphoric ones. In the second experiment, participants were asked to assess the meaningfulness of idiomatic, metaphoric and literal expressions after reading a verb prime that belongs to the target phrase (identity priming). The results showed that verb identity priming was stronger for idiomatic expressions than for metaphor ones, indicating different mental representations.     

How distraction affects pedestrian response: Evidence from behavior patterns and cortex oxyhemoglobin changes

ObjectivesDistracted walking is a major cause of pedestrian road traffic injuries, but little is known about how distraction affects pedestrian safety. The study was designed to explore how visual and auditory distraction might influence pedestrian safety.MethodsThree experiments were conducted to explore causal mechanisms from two theoretical perspectives, increased cognitive load from the distraction task and resource competition in the same sensory modality. Pedestrians’ behavior patterns and cortex oxyhemoglobin changes were recorded while they performed a series of dual tasks.ResultsFour primary results emerged: (a) participants responded more slowly to both visual and auditory stimuli in traffic, as well as walked more slowly, while talking on the phone or text messaging compared to when undistracted or listening to music; (b) when participants completed pedestrian response tasks while distracted with a high cognitive load, their response was significantly slower and poorer than when they carried out a lower cognitive load distraction task, (c) participants had higher levels of oxy-Hb change in cortices related to visual processing and executive function while distracted with a higher cognitive load; and (d) participants' responses to traffic lights were slower and resulted in a higher activation in prefrontal cortex and occipital areas when distracted by a visual distraction task compared to when distracted with an auditory task; similarly, brain activation increased significantly in temporal areas when participants responded to an auditory car horn task compared to when they responded to visual traffic lights.ConclusionsBoth distracting cognitive load demands and the type of distraction task significantly affect young adult pedestrian performance and threaten pedestrian safety. Pedestrian injury prevention efforts should consider the effects of the type of distracting task and its cognitive demands on pedestrian safety.     

The Effect of modality specific interference on working memory in recalling aversive auditory and visual memories*

Both auditory and visual emotional memories can be made less emotional by loading working memory (WM) during memory recall. Taxing WM during recall can be modality specific (giving an auditory [visuospatial] load during recall of an auditory [visual] memory) or cross modal (an auditory load during visual recall or vice versa). We tested whether modality specific loading taxes WM to a larger extent than cross modal loading. Ninety-six participants undertook a visual and auditory baseline Random Interval Repetition task (i.e. responding as fast as possible to a visual or auditory stimulus by pressing a button). Then, participants recalled a distressing visual and auditory memory, while performing the same visual and auditory Random Interval Repetition task. Increased reaction times (compared to baseline) were indicative of WM loading. Using Bayesian statistics, we compared five models in terms of general and modality specific taxation. There was support for the model describing the effect on WM of dual tasking in general, irrespective of modality specificity, and for the model describing the effect of modality specific loading. Both models combined gained the most support. The results suggest a general effect of dual tasking on taxing WM and a superimposed effect of taxing in matched modality.     

特征整合与加工深度对道德概念容器隐喻联结的影响

提出了隐喻提取假说将隐喻联结的形成和提取进行分离, 并通过3个实验探究了道德概念与容器空间的隐喻联结及其受知觉加工深度和特征整合程度的影响。实验1采用空间Stroop范式, 实验2a和实验2b均采用启动范式, 实验3a和实验3b均采用加入任务要求的Stroop范式。结果发现：(1)在经典Stroop范式中未发现道德概念与容器空间的隐喻联结; (2)在启动范式中发现, 较深知觉加工深度下道德概念与容器空间存在较弱的隐喻联结; (3)在较高特征整合程度的Stroop任务中, 道德概念与容器空间存在较强的隐喻联结。结果表明：道德概念与容器空间存在道德为内、不道德为外的隐喻联结, 这种隐喻联结在映射上表现为双向性, 并且受到特征整合程度和知觉加工深度的影响, 同时也为隐喻提取假说提供了证据支持。     

道德概念垂直空间隐喻对空间关系判断的影响

本研究通过三个实验考察道德概念垂直空间隐喻对空间关系判断的影响。涉及的空间关系判断包括上下关系判断、远近关系判断和距离判断。研究结果显示：（1）上下判断中,在空间上方时,道德词的反应快于不道德词;在空间下方时,不道德词的反应快于道德词;（2）远近判断中,在空间上方时,个体更倾向于将道德词判断为"远",即道德词更偏上;在空间下方时,没有显著的偏向;（3）在距离判断中,个体对道德词的判断出现显著的向上偏移,对不道德词的判断则出现显著的向下偏移。由此得出结论：道德概念的垂直空间隐喻会影响个体对空间关系的判断,具体来说是"道德是上"的隐喻会导致空间关系判断产生"向上"的偏移效应;而"不道德是下"的隐喻则会导致空间关系判断产生"向下"的偏移效应。     

Lexical orthographic knowledge acquisition in adults: The whole-word visual processing impact

IntroductionOne of the most persistent difficulties in French written language acquisition is lexical orthographic memorization. Both theoretical models and behavioral studies have suggested that simultaneous visual processing of all the letters of a word could be important for the acquisition of its orthographic form.Main goalTwo experiments are conducted to test this whole-word visual processing hypothesis.MethodThe paradigm used in both experiments is a self-teaching paradigm in which adult participants had to read orthographically complex bisyllabic pseudowords in isolation. In one reading condition, all the letters of the item are available at once, in the other the first and second syllables are seen successively. After reading, participants had to spell under dictation and to recognize the written items.ResultsGlobally, the results showed that participants better recall the orthographic form of a word after having read it in the whole-word reading condition. The result of the recognition task, in the second experiment, was in line with the result of the spelling under dictation task.ConclusionThese results, although they should be interpreted with caution, are in line with the whole-word visual processing hypothesis. Applied consequences for orthographic learning and teaching, as for remediation of specific orthographic disabilities, are discussed.     

Evidence for role-neutral initial processing of metaphors

Two models of metaphor processing are contrasted. The structure-mapping model postulates an initially role-neutral alignment process, followed by directional projection of inferences. The attributive categorization model postulates role-specific processing throughout comprehension. To test between these models, the early stages of metaphor comprehension were probed using a technique based on S. Glucksberg, P. Gildea, and H. Bookin's (1982) finding that metaphorical meaning interferes with literal truthfulness judgments. In Experiment 1, interference effects did not differ between normal metaphors and metaphors with reversed terms, suggesting that initial processing is role-neutral. In Experiment 2, we again found no role dependence in interference effects, even for highly conventional metaphors. In Experiment 3, it was verified that (a) full comprehension is role-sensitive and (b) full comprehension reaction times (RTs) are far longer than interference RTs, buttressing the claim that interference is an early-stage effect. Overall, the results support the structure-mapping model of metaphor processing.     

Suppressing visual feedback in written composition: Effects on processing demands and coordination of the writing processes

The goal of this experiment was to investigate the role of visual feedback during written composition. Effects of suppression of visual feedback were analyzed both on processing demands and on on‐line coordination of low‐level execution processes and of high‐level conceptual and linguistic processes. Writers composed a text and copied it either with or without visual feedback. Processing demands of the writing processes were evaluated with reaction times to secondary auditory probes, which were analyzed according to whether participants were handwriting (in a composing and a copying task) or engaged in high‐level processes (when pausing in a composing task). Suppression of visual feedback increased reaction time interference (secondary reaction time minus baseline reaction time) during handwriting in the copying task and not during pauses in the composing task. This suggests that suppression of visual feedback only affected processing demands of execution processes and not those of high‐level conceptual and linguistic processes. This is confirmed by analysis of the quality of the texts produced by participants, which were little, if at all, affected by the suppression of visual feedback. Results also indicate that the increase in processing demands of execution related to suppression of visual feedback affected on‐line coordination of the writing processes. Indeed, when visual feedback was suppressed, reaction time interferences associated with handwriting were not reliably different in the copying task or the composing task but were significantly different when visual feedback was not suppressed: They were lower in the copying task than in the composition task. When visual feedback was suppressed, writers activated step‐by‐step execution processes and high‐level writing processes, whereas they concurrently activated these writing processes when composing with visual feedback.     

实物形状的知觉相似性对视觉隐喻加工的影响

视觉隐喻是众多研究者比较关注的领域, 当前多以物体简笔画、漫画及广告等方面的研究为主, 但对与生活密切相关的实物的视觉隐喻研究甚少。因此研究采用掩蔽启动范式、李克特9点量表及语音生成等研究方法, 在物体概念类型(概念相同、概念相异)及形状类型(形状相似、形状相异)条件下, 针对实物图片的形状知觉相似性是否对视觉隐喻加工产生影响进行探讨。结果发现无论物体概念是否相同, 形状相似性对个体的物体识别具有“促进”作用, 并有利于视觉隐喻加工; 在物体识别中, 当两个物体概念不同时, 个体会试图建立一个特设类别来对形状相似的物体进行归类, 从而实现了视觉隐喻的加工。因此研究进一步证明了实物形状的知觉相似性有利于视觉隐喻的加工。     

Cognitive demands of error processing

This study used a dual-task methodology to assess attention demands associated with error processing during an anticipation-timing task. A difference was predicted in attention demands during feedback on trials with correct responses and errors. This was addressed by requiring participants to respond to a probe reaction-time stimulus after augmented feedback presentation. 16 participants (8 men, 8 women) completed two phases, the reaction time task only and the anticipation-timing task with the probe RT task. False feedback indicating error and a financial reward manipulation were used to increase relevance of errors. Data supported the hypothesis that error processing is associated with higher cognitive demands than processing feedback denoting a correct response. Individuals responded with quicker probe reaction times during presentation of feedback on correct trials than on error trials. These results are discussed with respect to the cognitive processes which might occur during error processing and their role in motor learning.     

Strategies or similarities—what determines mastery of figurative language?

Six experiments examined different determinants of metaphor performance. Special focus was placed on tenor-vehicle similarities and on visual-verbal strategy operations. The results showed that use of specified strategy operations tended to favour verbal operations, whereas previous research demonstrated that unspecified visual strategies improve metaphor comprehension. The deviating observations were interpreted as due to differences in levels of strategy specification. In the new experiments metaphors were shown to be less easily transformed under visual than under verbal processing conditions. The number of detected similarities between tenor and vehicle terms are shown to be only modestly related to metaphor performance. Findings suggested that similarities are more easily found with visually encoded metaphors, whereas an alternative tenor term is easier to find with the metaphor verbally encoded.     

Effects of verbal working memory deficits on metaphor comprehension in patients with Parkinson's disease

This research studied one aspect of pragmatic language processing, the ability to understand metaphorical language, to determine whether patients with Parkinson disease (PD) are impaired for these abilities, and whether cognitive resource limitations/fronto-striatal dysfunction contributes to these deficits. Seventeen PD participants and healthy controls (HC) completed a series of neuropsychological tests and performed a metaphor comprehension task following the methods of Gernsbacher and colleagues [Gernsbacher, M. A., Keysar, B., Robertson, R. R. W., & Werner, N. K. (2001). The role of suppression and enhancement in understanding metaphors. Journal of Memory and Language, 45, 433-450.] When participants in the PD group were identified as "impaired" or "unimpaired" relative to the control group on a measure of verbal working memory span, we found that only PD participants with impaired working memory were simultaneously impaired in the processing of metaphorical language. Based on our findings we argue that certain "complex" forms of language processing such as metaphor interpretation are highly dependent on intact fronto-striatal systems for working memory which are frequently, although not always, compromised during the early course of PD.     

Presentation format effects in working memory: The role of attention

Four experiments are reported in which participants attempted to remember three or six concrete nouns, presented as pictures, spoken words, or printed words, while also verifying the accuracy of sentences. Hypotheses meant to explain the higher recall of pictures and spoken words over printed words were tested. Increasing the difficulty and changing the type of processing task from arithmetic to a visual/spatial reasoning task did not influence recall. An examination of long-term modality effects showed that those effects were not sufficient to explain the superior performance with spoken words and pictures. Only when we manipulated the allocation of attention to the items in the storage task by requiring the participants to articulate the items and by presenting the stimulus items under a degraded condition were we able to reduce or remove the effect of presentation format. The findings suggest that the better recall of pictures and spoken words over printed words result from the fact that under normal presentation conditions, printed words receive less processing attention than pictures and spoken words do.     

汉语隐喻理解时程的ERPs研究

摘 要 本研究使用ERPs技术考察隐喻句与本义句理解时程的异同,以研究汉语隐喻的理解机制。以句尾范式向被试随机呈现本义句、熟悉隐喻句、新奇隐喻句及错误义句各50句,被试的任务为判断每句话是否有意义。结果显示：两种隐喻句诱发的N400波形与头皮分布情况和本义句无明显差异,且两种隐喻句诱发的N400波幅高于本义句。本研究结果证明汉语隐喻义和本义理解的认知机制类似,支持隐喻理解的平行加工假说,且加工隐喻义需要耗费更多的认知资源。     

Influence of a psychological perspective on scene viewing and memory for scenes

In the study, 33 participants viewed photographs from either a potential homebuyer's or a burglar's perspective, or in preparation for a memory test, while their eye movements were recorded. A free recall and a picture recognition task were performed after viewing. The results showed that perspective had rapid effects, in that the second fixation after the scene onset was more likely to land on perspective-relevant than on perspective-irrelevant areas within the scene. Perspective-relevant areas also attracted longer total fixation time, more visits, and longer first-pass dwell times than did perspective-irrelevant areas. As for the effects of visual saliency, the first fixation was more likely to land on a salient than on a nonsalient area; salient areas also attracted more visits and longer total fixation time than did nonsalient areas. Recall and recognition performance reflected the eye fixation results: Both were overall higher for perspective-relevant than for perspective-irrelevant scene objects. The relatively low error rates in the recognition task suggest that participants had gained an accurate memory for scene objects. The findings suggest that the role of bottom-up versus top-down factors varies as a function of viewing task and the time-course of scene processing.     

Influence of a psychological perspective on scene viewing and memory for scenes

In the study, 33 participants viewed photographs from either a potential homebuyer's or a burglar's perspective, or in preparation for a memory test, while their eye movements were recorded. A free recall and a picture recognition task were performed after viewing. The results showed that perspective had rapid effects, in that the second fixation after the scene onset was more likely to land on perspective-relevant than on perspective-irrelevant areas within the scene. Perspective-relevant areas also attracted longer total fixation time, more visits, and longer first-pass dwell times than did perspective-irrelevant areas. As for the effects of visual saliency, the first fixation was more likely to land on a salient than on a nonsalient area; salient areas also attracted more visits and longer total fixation time than did nonsalient areas. Recall and recognition performance reflected the eye fixation results: Both were overall higher for perspective-relevant than for perspective-irrelevant scene objects. The relatively low error rates in the recognition task suggest that participants had gained an accurate memory for scene objects. The findings suggest that the role of bottom-up versus top-down factors varies as a function of viewing task and the time-course of scene processing.