The impact of continuity editing in narrative film on event segmentation

Filmmakers use continuity editing to engender a sense of situational continuity or discontinuity at editing boundaries. The goal of this study was to assess the impact of continuity editing on how people perceive the structure of events in a narrative film and to identify brain networks that are associated with the processing of different types of continuity editing boundaries. Participants viewed a commercially produced film and segmented it into meaningful events, while brain activity was recorded with functional magnetic resonance imaging (MRI). We identified three degrees of continuity that can occur at editing locations: edits that are continuous in space, time, and action; edits that are discontinuous in space or time but continuous in action; and edits that are discontinuous in action as well as space or time. Discontinuities in action had the biggest impact on behavioral event segmentation, and discontinuities in space and time had minor effects. Edits were associated with large transient increases in early visual areas. Spatial-temporal changes and action changes produced strikingly different patterns of transient change, and they provided evidence that specialized mechanisms in higher order perceptual processing regions are engaged to maintain continuity of action in the face of spatiotemporal discontinuities. These results suggest that commercial film editing is shaped to support the comprehension of meaningful events that bridge breaks in low-level visual continuity, and even breaks in continuity of spatial and temporal location.     

Situation Models: The Mental Leap Into Imagined Worlds

Situation models are mental representations of the state of affairs described in a text rather than of the text itself. Much of the research on situation models in narrative comprehension suggests that comprehenders behave as though they are in the narrated situation rather than outside of it. This article reviews some of this evidence and provides an outlook on future developments.     

Walking through doorways causes forgetting: Further explorations

Previous research using virtual environments has revealed a location-updating effect in which there is a decline in memory when people move from one location to another. Here we assess whether this effect reflects the influence of the experienced context, in terms of the degree of immersion of a person in an environment, as suggested by some work in spatial cognition, or by a shift in context. In Experiment 1, the degree of immersion was reduced by using smaller displays. In comparison, in Experiment 2 an actual, rather than a virtual, environment was used, to maximize immersion. Location-updating effects were observed under both of these conditions. In Experiment 3, the original encoding context was reinstated by having a person return to the original room in which objects were first encoded. However, inconsistent with an encoding specificity account, memory did not improve by reinstating this context. Finally, we did a further analysis of the results of this and previous experiments to assess the differential influence of foregrounding and retrieval interference. Overall, these data are interpreted in terms of the event horizon model of event cognition and memory.     

Typhoid Pollyanna: The effect of category valence on retrieval order of positive and negative category members

Previous research suggests that people process pleasant information more efficiently than unpleasant information. This phenomenon has been illustrated in a variety of contexts and paradigms, and is frequently referred to as the Pollyanna principle. One important aspect of the Pollyanna principle is that people tend to retrieve pleasant members of a category from semantic memory prior to unpleasant members of the same category. However, we propose that the retrieval advantage of favourably viewed category members holds only for positive categories and that prototypicality of category members might be a better predictor of ease of retrieval than favourability. These possibilities were tested in two studies. In Study 1, favourability effects on retrieval order were observed for positive categories but not for neutral or negative categories. In Study 2, prototypical category members were retrieved before less prototypical category members for both positive and negative categories. These results suggest that the Pollyanna principle might be the result of a confound between favourability and prototypicality for members of favourable categories.     

The Wind Chilled the Spectators, but the Wine Just Chilled: Sense, Structure, and Sentence Comprehension

Anticipation plays a role in language comprehension. In this article, we explore the extent to which verb sense influences expectations about upcoming structure. We focus on change of state verbs like shatter , which have different senses that are expressed in either transitive or intransitive structures, depending on the sense that is used. In two experiments we influence the interpretation of verb sense by manipulating the thematic fit of the grammatical subject as cause or affected entity for the verb, and test whether readers' expectations for a transitive or intransitive structure change as a result. This sense-biasing context influenced reading times in the postverbal regions. Reading times for transitive sentences were faster following good-cause than good-theme subjects, but the opposite pattern was found for intransitive sentences. We conclude that readers use sense-contingent subcategorization preferences during on-line comprehension.     

Perception of auditory motion affects language processing

Previous reports have demonstrated that the comprehension of sentences describing motion in a particular direction (toward, away, up, or down) is affected by concurrently viewing a stimulus that depicts motion in the same or opposite direction. We report 3 experiments that extend our understanding of the relation between perception and language processing in 2 ways. First, whereas most previous studies of the relation between perception and language processing have focused on visual perception, our data show that sentence processing can be affected by the concurrent processing of auditory stimuli. Second, it is shown that the relation between the processing of auditory stimuli and the processing of sentences depends on whether the sentences are presented in the auditory or visual modality.     

From demonstration to theory in embodied language comprehension: A review

Recent findings in psychology, psycholinguistics, and neuroscience present a challenge to current amodal theories by suggesting that cognitive states are not disembodied in language comprehension. Accumulating behavioral evidence supporting this view is reviewed from research on processing of language describing concrete and abstract concepts. The extant embodied theories that support either a strong or a moderate embodied view are then presented, as are the perspectives that define how the researchers discuss the role of sensory-motor grounding in language processing. The article concludes by discussing several lines of research that might help distinguish between various theoretical approaches and resolve some of the fundamental issues that fuel much of the debate in the field.     

Semantic Memory Search and Retrieval in a Novel Cooperative Word Game: A Comparison of Associative and Distributional Semantic Models

Considerable work during the past two decades has focused on modeling the structure of semantic memory, although the performance of these models in complex and unconstrained semantic tasks remains relatively understudied. We introduce a two-player cooperative word game, Connector (based on the boardgame Codenames), and investigate whether similarity metrics derived from two large databases of human free association norms, the University of South Florida norms and the Small World of Words norms, and two distributional semantic models based on large language corpora (word2vec and GloVe) predict performance in this game. Participant dyads were presented with 20-item word boards with word pairs of varying relatedness. The speaker received a word pair from the board (e.g., exam-algebra) and generated a one-word semantic clue (e.g., math), which was used by the guesser to identify the word pair on the board across three attempts. Response times to generate the clue, as well as accuracy and latencies for the guessed word pair, were strongly predicted by the cosine similarity between word pairs and clues in random walk-based associative models, and to a lesser degree by the distributional models, suggesting that conceptual representations activated during free association were better able to capture search and retrieval processes in the game. Further, the speaker adjusted subsequent clues based on the first attempt by the guesser, who in turn benefited from the adjustment in clues, suggesting a cooperative influence in the game that was effectively captured by both associative and distributional models. These results indicate that both associative and distributional models can capture relatively unconstrained search processes in a cooperative game setting, and Connector is particularly suited to examine communication and semantic search processes.     

Discourse-mediation of the mapping between language and the visual world: Eye movements and mental representation

Two experiments explored the mapping between language and mental representations of visual scenes. In both experiments, participants viewed, for example, a scene depicting a woman, a wine glass and bottle on the floor, an empty table, and various other objects. In Experiment 1, participants concurrently heard either ‘The woman will put the glass on the table’ or ‘The woman is too lazy to put the glass on the table’. Subsequently, with the scene unchanged, participants heard that the woman ‘will pick up the bottle, and pour the wine carefully into the glass.’ Experiment 2 was identical except that the scene was removed before the onset of the spoken language. In both cases, eye movements after ‘pour’ (anticipating the glass) and at ‘glass’ reflected the language-determined position of the glass, as either on the floor, or moved onto the table, even though the concurrent (Experiment 1) or prior (Experiment 2) scene showed the glass in its unmoved position on the floor. Language-mediated eye movements thus reflect the real-time mapping of language onto dynamically updateable event-based representations of concurrently or previously seen objects (and their locations).     

Walking through doorways causes forgetting: recall

The aim of the current study was to explore how the location updating effect is affected when people are tested using recall rather than recognition, which is what has been done in prior work. Differences in the memory processes involved with these two tasks lead to predictions for two different patterns of data. In Experiment 1, memory was tested by having participants recall the single object they were carrying or had just put down, whereas in Experiment 2, people sometimes needed to recall both objects. It was found that, unlike recognition test performance, a similar location updating effect was found for both Associated (what was currently being carried) and Dissociated (what was recently set down) objects. Moreover, when both objects were correctly recalled, there was a bias to remember them in the order that they were encountered. Finally, if only one object was correctly recalled, it was the Associated object that was currently being carried. Overall, these results are consistent with the Event Horizon Model of event cognition.     

The role of illustrations in children’s inferential comprehension

Illustrations are a salient source of information in children’s books, yet their effect on children’s reading comprehension has been studied only through literal factual recall. The purpose of the current study was to determine the effect of illustrations on bridging inferences, an important aspect of meaning making in comprehension models. Identical short stories were presented under different illustration conditions with pictures that represented different parts of the story. Participants were 73 7- to 11-year-olds. Illustrations both facilitated and interfered with inferencing depending on the type of information depicted; however, this effect was reduced as grade increased. Additional findings were that the overall ability to make inferences increased with age and working memory was a significant predictor of this skill. Results are discussed in relation to cognitive and developmental models of comprehension.     

Why we simulate negated information: A dynamic pragmatic account

A well-established finding in the simulation literature is that participants simulate the positive argument of negation soon after reading a negative sentence, prior to simulating a scene consistent with the negated sentence (Kaup, Lüdtke, & Zwaan, 2006 Kaup, B., Lüdtke, J. and Zwaan, R. A. 2006. Processing negated sentences with contradictory predicates: Is a door that is not open mentally closed?. Journal of Pragmatics, 38: 1033–1050. [Crossref], [Web of Science ®] , [Google Scholar]; Kaup, Yaxley, Madden, Zwaan, & Lüdtke, 2007 Kaup, B., Yaxley, R. H., Madden, C. J., Zwaan, R. A. and Lüdtke, J. 2007. Experiential simulations of negated text information. Quarterly Journal of Experimental Psychology, 60: 976–990. [Taylor & Francis Online], [Web of Science ®] , [Google Scholar]). One interpretation of this finding is that negation requires two steps to process: first represent what is being negated then “reject” that in favour of a representation of a negation-consistent state of affairs (Kaup et al., 2007 Kaup, B., Yaxley, R. H., Madden, C. J., Zwaan, R. A. and Lüdtke, J. 2007. Experiential simulations of negated text information. Quarterly Journal of Experimental Psychology, 60: 976–990. [Taylor & Francis Online], [Web of Science ®] , [Google Scholar]). In this paper we argue that this finding with negative sentences could be a by-product of the dynamic way that language is interpreted relative to a common ground and not the way that negation is represented. We present a study based on Kaup et al. (2007) Kaup, B., Yaxley, R. H., Madden, C. J., Zwaan, R. A. and Lüdtke, J. 2007. Experiential simulations of negated text information. Quarterly Journal of Experimental Psychology, 60: 976–990. [Taylor & Francis Online], [Web of Science ®] , [Google Scholar] that tests the competing accounts. Our results suggest that some negative sentences are not processed in two steps, but provide support for the alternative, dynamic account.     

Moving words: dynamic representations in language comprehension

Eighty-two participants listened to sentences and then judged whether two sequentially presented visual objects were the same. On critical trials, participants heard a sentence describe the motion of a ball toward or away from the observer (e.g., “The pitcher hurled the softball to you”). Seven hundred and fifty milliseconds after the offset of the sentence, a picture of an object was presented for 500 ms, followed by another picture. On critical trials, the two pictures depicted the kind of ball mentioned in the sentence. The second picture was displayed 175 ms after the first. Crucially, it was either slightly larger or smaller than the first picture, thus suggesting movement of the ball toward or away from the observer. Participants responded more quickly when the implied movement of the balls matched the movement described in the sentence. This result provides support for the view that language comprehension involves dynamic perceptual simulations.     

The Action-Sentence Compatibility Effect: It's All in the Timing

When participants are asked to make sensibility judgments on sentences that describe action toward the body (i.e., "Mark dealt the cards to you") or away from the body (i.e., "You dealt the cards to Mark"), they are faster to respond when the response requires an arm movement in the same direction as the action described by the sentence. This congruence effect is known as the Action–Sentence Compatibility Effect (ACE). This study reports 4 experiments that extend our understanding of the ACE by exploring how the time at which one prepares the motor response required for the sensibility judgment affects the magnitude of the ACE. Results show that the ACE arises only when participants have the opportunity to plan their motor response while they are processing the sentence.     

The Fabric of Thought: Priming Tactile Properties During Reading Influences Direct Tactile Perception

The present studies examined whether implied tactile properties during language comprehension influence subsequent direct tactile perception, and the specificity of any such effects. Participants read sentences that implicitly conveyed information regarding tactile properties (e.g., Grace tried on a pair of thick corduroy pants while shopping) that were either related or unrelated to fabrics and varied in implied texture (smooth, medium, rough). After reading each sentence, participants then performed an unrelated rating task during which they felt and rated the texture of a presented fabric. Results demonstrated that the texture properties implied in sentences influence direct tactile perception. Specifically, after reading about a smooth or rough texture, subsequent fabric ratings became notably smoother or rougher, respectively. However, we also show that there was some specificity to these effects: Fabric‐related sentences elicited more specific and interactive effects on subsequent ratings. Together, we demonstrate that under certain circumstances, language comprehension can prime tactile representations and affect direct tactile perception. Results are discussed with regard to the nature and scope of multimodal mental simulation during reading.     

句子理解中颜色信息的心理表征

采用混合实验设计、实时和事后的句子-图片匹配范式, 以隐含物体典型颜色和非典型颜色信息的句子为实验材料, 以被试对图片的反应时间和阅读时间为因变量指标, 通过不同时间间隔的设置以及不同的实验程序, 探讨了句子理解中静态和动态颜色信息心理表征的特点。结果表明：(1)在加工时间有限的情况下, 两个加工任务是否竞争相同的认知资源是造成句-图范式下匹配易化和不匹配易化的关键因素。(2)对于句子隐含的静态颜色信息, 大脑对典型颜色信息的心理表征具有即时性和局部性, 而对非典型颜色信息的心理表征还具有非局部性的特点。(3)对于句子隐含的动态颜色信息, 大脑不能对其进行即时的心理表征, 这种动态颜色信息的心理表征是在句子阅读晚期发生的。