Language, perception, and the schematic representation of spatial relations

Schemas are abstract nonverbal representations that parsimoniously depict spatial relations. Despite their ubiquitous use in maps and diagrams, little is known about their neural instantiation. We sought to determine the extent to which schematic representations are neurally distinguished from language on the one hand, and from rich perceptual representations on the other. In patients with either left hemisphere damage or right hemisphere damage, a battery of matching tasks depicting categorical spatial relations was used to probe for the comprehension of basic spatial concepts across distinct representational formats (words, pictures, and schemas). Left hemisphere patients underperformed right hemisphere patients across all tasks. However, focused residual analyses using voxel-based lesion-symptom mapping (VLSM) suggest that (1) left hemisphere deficits in the representation of categorical spatial relations are difficult to distinguish from deficits in naming these relations and (2) the right hemisphere plays a special role in extracting schematic representations from richly textured pictures.     

A comparison of backward and forward spatial spans

The standard Corsi blocks task is frequently used to measure the capacity of visuospatial working memory, but the implications of the use of both forward and backward recall are still unclear. In the present study, we showed that the backward Corsi task is particularly powerful in discriminating between low- and high-spatial-ability individuals and involves different processes from those involved in the forward task. From a sample of 425 participants we selected one group of 20 high-spatial-ability participants and one of 20 low-spatial-ability participants. The results demonstrated that a backward spatial span offers specific information not available from a forward spatial span, and that there was no facilitation due to a descending format. In particular, in the low-spatial-ability group, performance was generally poorer, but backward Corsi recall was lower than forward recall, and participants did not show any advantage following the descending presentation format—which in some contexts is considered to reduce proactive interference. We conclude that the backward Corsi task has specific value and that the assumption of fully parallel verbal and visuospatial working-memory systems can lead to a variety of misunderstandings.     

小学生视觉-空间表征类型和数学问题解决的研究

本研究考察并比较了四至六年级儿童的视觉－空间表征策略、数学问题解决和空间视觉化能力。结果表明：五、六年级儿童的解题正确率、使用图式表征策略的程度显著高于四年级儿童;使用图像表征策略的程度各年级无显著差异。将数学问题分成三个难度等级,发现年级差异主要表现在难度等级1的题目上。另外,六年级儿童的空间视觉化能力显著高于四年级儿童。     

Object-position binding in visual memory for natural scenes and object arrays

Nine experiments examined the means by which visual memory for individual objects is structured into a larger representation of a scene. Participants viewed images of natural scenes or object arrays in a change detection task requiring memory for the visual form of a single target object. In the test image, 2 properties of the stimulus were independently manipulated: the position of the target object and the spatial properties of the larger scene or array context. Memory performance was higher when the target object position remained the same from study to test. This same-position advantage was reduced or eliminated following contextual changes that disrupted the relative spatial relationships among contextual objects (context deletion, scrambling, and binding change) but was preserved following contextual change that did not disrupt relative spatial relationships (translation). Thus, episodic scene representations are formed through the binding of objects to scene locations, and object position is defined relative to a larger spatial representation coding the relative locations of contextual objects.     

数学学习不良儿童视觉-空间表征与数学问题解决

采用临床访谈的方法,考察了30名数学学习不良(MD)儿童和31名一般儿童的数学问题解决、视觉-空间表征策略和空间视觉化能力。结果发现：图式表征能促进数学问题的解决,图像表征则起妨碍作用;空间视觉化能力与解题正确率及图式表征策略有显著正相关,与图像表征策略有显著负相关。MD儿童的解题正确率以及使用图式表征策略的程度显著低于一般儿童,使用图像表征策略的程度则显著高于一般儿童。在解题正确率和图式表征策略这两个变量上,MD儿童和一般儿童的年级发展趋势相同,都随年级的升高而提高。但在图像表征策略的使用上,一般儿童有随年级的升高而下降的趋势,MD儿童却没有下降的趋势。两类儿童的空间视觉化能力都随年级的升高而提高。     

动词理解中空间表征的激活过程

探讨动词理解中空间表征的激活过程。被试听以不同空间轴向动词作谓语的句子后辨认视觉刺激的形状。实验1以肯定句为材料探讨语义理解对视知觉任务的影响。实验2以客观原因否定句为材料排除句子表征的影响。实验3以主观原因否定句为材料探讨动词空间元素的激活机制。总的结果表明,动词理解过程中激活其表征中的空间元素,这种激活是自动的、非策略性的,不受情境中客观原因或主观意愿否定的影响。动词理解空间效应反映了语言表征中的知觉运动特征     

Space in Hand and Mind: Gesture and Spatial Frames of Reference in Bilingual Mexico

Speakers of many languages prefer allocentric frames of reference (FoRs) when talking about small-scale space, using words like “east” or “downhill.” Ethnographic work has suggested that this preference is also reflected in how such speakers gesture. Here, we investigate this possibility with a field experiment in Juchitán, Mexico. In Juchitán, a preferentially allocentric language (Isthmus Zapotec) coexists with a preferentially egocentric one (Spanish). Using a novel task, we elicited spontaneous co-speech gestures about small-scale motion events (e.g., toppling blocks) in Zapotec-dominant speakers and in balanced Zapotec-Spanish bilinguals. Consistent with prior claims, speakers’ spontaneous gestures reliably reflected either an egocentric or allocentric FoR. The use of the egocentric FoR was predicted—not by speakers’ dominant language or the language they used in the task—but by mastery of words for “right” and “left,” as well as by properties of the event they were describing. Additionally, use of the egocentric FoR in gesture predicted its use in a separate nonlinguistic memory task, suggesting a cohesive cognitive style. Our results show that the use of spatial FoRs in gesture is pervasive, systematic, and shaped by several factors. Spatial gestures, like other forms of spatial conceptualization, are thus best understood within broader ecologies of communication and cognition.     

Motion categorisation: Representing velocity qualitatively

Categorising is arguably one of the first steps in cognition, because it enables high-level cognitive processing. For a similar reason, categorising is a first step—a preprocessing step—in artificial intelligence, specifically in decision-making, reasoning, and natural language processing. In this paper we categorise the motion of entities. Such categorisations, also known as qualitative representations, represent the preprocessing step for navigation problems with dynamical obstacles. As a central result, we present a general method to generate categorisations of motion based on categorisations of space. We assess its general validity by generating two categorisations of motion from two different spatial categorisations. We show examples of how the categorisations of motion describe and control trajectories. We also establish its soundness in cognitive and mathematical principles.     

3~5岁儿童理解和使用空间表征的特点

以96名3~5儿童为被试,采用自编的儿童空间表征实验任务,在语言表征、模型表征和图画表征三种空间表征形式上,考察了儿童理解和使用空间表征的发展特点。结果表明:(1)总体上,3~5岁儿童理解空间表征的发展水平均显著高于使用空间表征的发展水平。(2)3~5岁儿童理解和使用空间表征的发展表现出显著的年龄效应。(3)从不同的空间表证形式来看,3岁儿童理解语言表征的能力与理解模型表征的能力之间差异显著,4岁儿童使用语言表征的能力与使用模型表征的能力之间的差异、使用语言表征的能力与使用图画表征的能力之间的差异、以及使用模型表征的能力与使用图画表征的能力之间的差异都显著,其余形式的空间表征理解之间的差异或空间表征使用之间的差异在各年龄段中均不显著。     

Solving small spaces: investigating the use of landmark cues in brown capuchins (Cebus apella)

Some researchers have recently argued that humans may be unusual among primates in preferring to use landmark information when reasoning about some kinds of spatial problems. Some have explained this phenomenon by positing that our species’ tendency to prefer landmarks stems from a human-unique trait: language. Here, we test this hypothesis—that preferring to use landmarks to solve such tasks is related to language ability—by exploring landmark use in a spatial task in one non-human primate, the brown capuchin monkey (Cebus apella). We presented our subjects with the rotational displacement task, in which subjects attempt to relocate a reward hidden within an array of hiding locations which are subsequently rotated to a new position. Over several experiments, we varied the availability and the salience of a landmark cue within the array. Specifically, we varied (1) visual access to the array during rotation, (2) the type of landmark, (3) the consistency of the landmark qualities, and (4) the amount of exposure to the landmark. Across Experiments 1 through 4, capuchins did not successfully use landmarks cues, suggesting that non-linguistic primates may not spontaneously use landmarks to solve some spatial problems, as in this case of a small-scale dynamic spatial task. Importantly, we also observed that capuchins demonstrated some capacity to learn to use landmarks in Experiment 4, suggesting that non-linguistic creatures may be able to use some landmarks cues in similar spatial tasks with extensive training.     

Does scene context always facilitate retrieval of visual object representations?

An object-to-scene binding hypothesis maintains that visual object representations are stored as part of a larger scene representation or scene context, and that scene context facilitates retrieval of object representations (see, e.g., Hollingworth, Journal of Experimental Psychology: Learning, Memory and Cognition, 32, 58-69, 2006). Support for this hypothesis comes from data using an intentional memory task. In the present study, we examined whether scene context always facilitates retrieval of visual object representations. In two experiments, we investigated whether the scene context facilitates retrieval of object representations, using a new paradigm in which a memory task is appended to a repeated-flicker change detection task. Results indicated that in normal scene viewing, in which many simultaneous objects appear, scene context facilitation of the retrieval of object representations-henceforth termed object-to-scene binding-occurred only when the observer was required to retain much information for a task (i.e., an intentional memory task).     

Representational momentum,spatial layout,and viewpoint dependency

Completing a representational momentum (RM) task, participants viewed one of three camera motions through a scene and indicated whether test probes were in the same position as they were in the final view of the animation. All camera motions led to RM anticipations in the direction of motion, with larger distortions resulting from rotations than a compound motion of a rotation and a translation. A surprise test of spatial layout, using an aerial map, revealed that the correct map was identified only following aerial views during the RM task. When the RM task displayed field views, including repeated views of multiple object groups, participants were unable to identify the overall spatial layout of the scene. These results suggest that the object–location binding thought to support certain change detection and visual search tasks might be viewpoint dependent.     

Using selective interference to investigate spatial memory representations

Two experiments used a selective interference procedure in an attempt to determine whether nonverbal visual stimuli were represented in memory in a verbal or spatial format. A spatial representation was clearly implicated. In both experiments, Ss were required to remember either the positions or the identities of seven target items in a 25-item array. During the retention interval for that information, Ss attempted to recognize schematic face or airplane photograph stimuli in a same-different memory task. Memory performance on one or both tasks was greatly impaired when the recall task involved position or spatial information, but was either much less or not at all affected by an identity or verbal information recall task. Because of the selective nature of the interference and on the basis of certain correlational evidence, the experimental results were also interpreted as providing support for the notion that verbal and spatial information are stored and processed in separate information-processing systems.     

Implicit scene learning is viewpoint dependent

When novel scenes are encoded, the representations of scene layout are generally viewpoint specific. Past studies of scene recognition have typically required subjects to explicitly study and encode novel scenes, but in everyday visual experience, it is possible that much scene learning occurs incidentally. Here, we examine whether implicitly encoded scene layouts are also viewpoint dependent. We used the contextual cuing paradigm, in which search for a target is facilitated by implicitly learned associations between target locations and novel spatial contexts (Chun & Jiang, 1998). This task was extended to naturalistic search arrays with apparent depth. To test viewpoint dependence, the viewpoint of the scenes was varied from training to testing. Contextual cuing and, hence, scene context learning decreased as the angular rotation from training viewpoint increased. This finding suggests that implicitly acquired representations of scene layout are viewpoint dependent.     

The effect of spatial and nonspatial contextual information on visual object memory

Recent research has found visual object memory can be stored as part of a larger scene representation rather than independently of scene context. The present study examined how spatial and nonspatial contextual information modulate visual object memory. Two experiments tested participants’ visual memory by using a change detection task in which a target object's orientation was either the same as it appeared during initial viewing or changed. In addition, we examined the effect of spatial and nonspatial contextual manipulations on change detection performance. The results revealed that visual object representations can be maintained reliably after viewing arrays of objects. Moreover, change detection performance was significantly higher when either spatial or nonspatial contextual information remained the same in the test image. We concluded that while processing complex visual stimuli such as object arrays, visual object memory can be stored as part of a comprehensive scene representation, and both spatial and nonspatial contextual changes modulate visual memory retrieval and comparison.     

Development of allocentric spatial recall from new viewpoints in virtual reality

Using landmarks and other scene features to recall locations from new viewpoints is a critical skill in spatial cognition. In an immersive virtual reality task, we asked children 3.5–4.5 years old to remember the location of a target using various cues. On some trials they could use information from their own self‐motion. On some trials they could use a view match. In the very hardest kind of trial, they were ‘teleported’ to a new viewpoint and could only use an allocentric spatial representation. This approach provides a strict test for allocentric coding (without either a matching viewpoint or self‐motion information) while avoiding additional task demands in previous studies (it does not require them to deal with a small table‐top environment or to manage stronger cue conflicts). Both the younger and older groups were able to point back at the target location better than chance when they could use view matching and/or self‐motion, but allocentric recall was only seen in the older group (4.0–4.5). In addition, we only obtained evidence for a specific kind of allocentric recall in the older group: they tracked one major axis of the space significantly above chance, r(158) = .28, but not the other, r(158) = ?.01. We conclude that there is a major qualitative change in coding for spatial recall around the fourth birthday, potentially followed by further development towards fully flexible recall from new viewpoints.     

Vision Using Routines: A Functional Account of Vision

This paper presents the case for a functional account of vision. A variety of studies have consistently revealed “change blindness” or insensitivity to changes in the visual scene during an eye movement. These studies indicate that only a small part of the information in the scene is represented in the brain from moment to moment. It is still unclear, however, exactly what is included in visual representations. This paper reviews experiments using an extended visuo-motor task, showing that display changes affect performance differently depending on the observer's place in the task. These effects are revealed by increases in fixation duration following a change. Different task-dependent increases suggest that the visual system represents only the information that is necessary for the immediate visual task. This allows a principled exploration of the stimulus properties that are included in the internal visual representation. The task specificity also has a more general implication that vision should be conceptualized as an active process executing special purpose “routines” that compute only the currently necessary information. Evidence for this view and its implications for visual representations are discussed. Comparison of the change blindness phenomenon and fixation durations shows that conscious report does not reveal the extent of the representations computed by the routines.     

The long and the short of it: On the nature and origin of functional overlap between representations of space and time

When we describe time, we often use the language of space (The movie was long; The deadline is approaching). Experiments 1–3 asked whether—as patterns in language suggest—a structural similarity between representations of spatial length and temporal duration is easier to access than one between length and other dimensions of experience, such as loudness. Adult participants were shown pairings of lines of different length with tones of different duration (Experiment 1) or tones of different loudness (Experiment 2). The length of the lines and duration or loudness of the tones was either positively or negatively correlated. Participants were better able to bind particular lengths and durations when they were positively correlated than when they were not, a pattern not observed for pairings of lengths and tone amplitudes, even after controlling for the presence of visual cues to duration in Experiment 1 (Experiment 3). This suggests that representations of length and duration may functionally overlap to a greater extent than representations of length and loudness. Experiments 4 and 5 asked whether experience with and mastery of words like long and short—which can flexibly refer to both space and time—itself creates this privileged relationship. Nine-month-old infants, like adults, were better able to bind representations of particular lengths and durations when these were positively correlated (Experiment 4), and failed to show this pattern for pairings of lengths and tone amplitudes (Experiment 5). We conclude that the functional overlap between representations of length and duration does not result from a metaphoric construction processes mediated by learning to flexibly use words such as long and short. We suggest instead that it may reflect an evolutionary recycling of spatial representations for more general purposes.