Is Language Required to Represent Others’ Mental States? Evidence From Beliefs and Other Representations

An important part of our Theory of Mind—the ability to reason about other people's unobservable mental states—is the ability to attribute false beliefs to others. We investigated whether processing these false beliefs, as well as similar but nonmental representations, is reliant on language. Participants watched videos in which a protagonist hides a gift and either takes a photo of it or writes a text about its location before a second person inadvertently moves the present to a different location, thereby rendering the belief and either the photo or text false. At the same time, participants performed either a concurrent verbal interference task (rehearsing strings of digits) or a visual interference task (remembering a visual pattern). Results showed that performance on false belief trials did not decline under verbal interference relative to visual interference. We interpret these findings as further support for the view that language does not form an essential part of the process of reasoning online (“in the moment”) about false beliefs.     

Centre-embedded structures are a by-product of associative learning and working memory constraints: evidence from baboons (Papio Papio)

Influential theories have claimed that the ability for recursion forms the computational core of human language faculty distinguishing our communication system from that of other animals (Hauser, Chomsky, & Fitch, 2002). In the present study, we consider an alternative view on recursion by studying the contribution of associative and working memory processes. After an intensive paired-associate training with visual shapes, we observed that baboons spontaneously ordered their responses in keeping with a recursive, centre-embedded structure. This result suggests that the human ability for recursion might partly if not entirely originate from fundamental processing constraints already present in nonhuman primates and that the critical distinction between animal communication and human language should more likely be found in working memory capacities than in an ability to produce recursive structures per se.     

Effects of television preplay formats on children's attention and story comprehension

Childrens' visual attention to, and comprehension of, a television program was measured as a fucntion of inserts called preplays that varied on two orthogoal dimensions: (1) Presence or absence of visual excerpts from the program, and (2) concrete or inferential story narration. Visual fixation was coded continously for 64 pairs of same-sex children, in 1st 4th grades, while they viewed the television program with one of four types of prepalys. After veiwing, each child answered items assessing his or her comprehension of the visual and verbally presented content. Children who viewed visual preplays attended longer than did children who viewed nonvisual preplays. Visual presentation predicted comprehension of content presented in a visual mode, whereas inferential narration predicted comprehension of implicit content presented in a verbal mode. The results suggest that information processing is modality specific: Visual presentation affects visual processing and abstract language affects verbal processing. The results do not suport the hypothesis that visual presentation interfere with linguistic processing.     

Measurements of auditory-verbal STM span in aphasia: effects of item, task, and lexical impairment

In the first part of this study, we investigated effects of item and task type on span performance in a group of aphasic individuals with word processing and STM deficits. Group analyses revealed significant effects of item on span performance with span being greater for digits than for words. We also investigated associations between subjects' lexical-semantic and phonological processing abilities and performance on four measures of verbal span (digit and word span, each varied for type of response, verbal vs. pointing) as well as one measure of nonverbal span. We predicted and found that the patterns of association between verbal span tasks and lexical abilities reflected the integrity of language processes and representations deployed in each paradigm used to assess span. Performance on the pointing span task, which engages both lexical-semantic and phonological processes, correlated with measures of both lexical-semantic and phonological abilities. Performance on repetition span, which engages primarily input and output phonological processes, correlated with measures of phonological abilities but not measures of lexical-semantic abilities. However, when partial correlations were performed for two subject groups based on their relative preservation of lexical-semantic ability (less or more than phonological ability), repetition span correlated with lexical-semantic measures only in the subgroup with relatively impaired lexical-semantics. Additionally, performance on the nonverbal span task correlated with measures of phonological abilities, suggesting either a general cognitive deficit affecting verbal and nonverbal STM or possibly, the use of a verbal strategy to perform this task. Our discussion focuses on the interpretation of span measurements in clinical practice and research, as well as the implications of these data for theories of short-term memory and word processing.     

Verbal, visual, and spatial working memory in written language production

College students wrote definitions of either abstract or concrete nouns in longhand while performing a concurrent working memory (WM) task. They detected either a verbal (syllable), visual (shape), or spatial (location) stimulus and decided whether it matched the last one presented 15-45s earlier. Writing definitions of both noun types elevated the response time to verbal targets above baseline. Such interference was observed for visual targets only when defining concrete nouns and was eliminated entirely with spatial targets. The interference effect for verbal targets was the same whether they were read or heard, implicating phonological storage. The findings suggest that language production requires phonological or verbal WM. Visual WM is selectively engaged when imaging the referents of concrete nouns.     

工作记忆负荷、形状干扰对聋人与听力正常人注意捕获影响的眼动研究

通过记录和分析被试的行为和眼动指标,本研究试图探讨不同工作记忆负荷下,形状干扰对聋人与听力正常人不同视野位置上刺激信息捕获注意的异同。研究结果表明：（1）形状干扰刺激虽然没有影响被试视觉加工的认知绩效,但被试的眼动模式会由于工作记忆负荷的改变而改变,高工作记忆负荷时,有形状干扰的注视次数高于无形状干扰。（2）工作记忆负荷、视野位置影响被试视觉加工任务的完成,被试在不同视野位置的加工过程表现出不一致的眼动模式。（3）聋人视觉加工在高工作记忆负荷时存在一定的劣势,但其对不同视野位置的信息加工及形状干扰所产生的注意捕获效应与听力正常人一致。     

Individual differences in language development: relationship with motor skill at 21 months

Language development has long been associated with motor development, particularly manual gesture. We examined a variety of motor abilities – manual gesture including symbolic, meaningless and sequential memory, oral motor control, gross and fine motor control – in 129 children aged 21 months. Language abilities were assessed and cognitive and socio‐economic measures controlled for. Oral motor control was strongly associated with language production (vocabulary and sentence complexity), with some contribution from symbolic abilities. Language comprehension, however, was associated with cognitive and socio‐economic measures. We conclude that symbolic, working memory, and mirror neuron accounts of language–motor control links are limited, but that a common neural and motor substrate for nonverbal and verbal oral movements may drive the motor–language association.     

Visual perceptual abilities of Chinese-speaking and English-speaking children

This paper reports an investigation of Chinese-speaking and English-speaking children's general visual perceptual abilities. The Developmental Test of Visual Perception was administered to 41 native Chinese-speaking children of mean age 5 yr. 4 mo. in Hong Kong and 35 English-speaking children of mean age 5 yr. 2 mo. in Melbourne. Of interest were the two interrelated components of visual perceptual abilities, namely, motor-reduced visual perceptual and visual-motor integration perceptual abilities, which require either verbal or motoric responses in completing visual tasks. Chinese-speaking children significantly outperformed the English-speaking children on general visual perceptual abilities. When comparing the results of each of the two different components, the Chinese-speaking students' performance on visual-motor integration was far better than that of their counterparts (ES = 2.70), while the two groups of students performed similarly on motor-reduced visual perceptual abilities. Cultural factors such as written language format may be contributing to the enhanced performance of Chinese-speaking children's visual-motor integration abilities, but there may be validity questions in the Chinese version.     

Toward a Connectionist Model of Recursion in Human Linguistic Performance

Naturally occurring speech contains only a limited amount of complex recursive structure, and this is reflected in the empirically documented difficulties that people experience when processing such structures. We present a connectionist model of human performance in processing recursive language structures. The model is trained on simple artificial languages. We find that the qualitative performance profile of the model matches human behavior, both on the relative difficulty of center-embedding and cross-dependency, and between the processing of these complex recursive structures and right-branching recursive constructions. We analyze how these differences in performance are reflected in the internal representations of the model by performing discriminant analyses on these representations both before and after training. Furthermore, we show how a network trained to process recursive structures can also generate such structures in a probabilistic fashion. This work suggests a novel explanation of people's limited recursive performance, without assuming the existence of a mentally represented competence grammar allowing unbounded recursion.     

Working memory and mental arithmetic: A case for dual central executive resources

The current study was designed to examine the possible existence of two limited-capacity pools of central executive resources: one each for verbal and visuospatial processing. Ninety-one college students (M age = 19.0, SD = 2.2) were administered a verbal working memory task that involved updating numbers in 2-, 3-, and 4-load conditions. The task was administered in both single task (no-interference condition) and dual-task (verbal interference and visuospatial interference conditions) formats. Findings indicated main effects for both memory load and type of interference, as well as, a load × interference interaction. Verbal interference led to a steeper increase in errors on the primary verbal working memory task; whereas, there was a smaller increase in errors across load in both the non-verbal and no-interference conditions. The effect of verbal interference and the lack of a spatial interference effect on a primary task that utilized verbal working memory resources, suggests that the processing of verbal and spatial stimuli in a dual-task paradigm requires separate central executive resources.     

The effects of verbal and visual interference on impressions: An associated-systems approach

This study investigated associated-systems theory's prediction that interference with the visual or verbal mental systems should alter person impressions by removing or reducing the impact of the information represented within those systems, while accentuating the influence of other information. In Experiment 1, interference with the verbal system decreased the influence of verbal information on trait judgments and increased the influence of visual information on attractiveness and liking judgments. In Experiment 2, visual interference decreased the impact of visual information on attractiveness judgments and, at least for attractive targets, increased the impact of verbal information on trait and liking judgments. These results support the premise that impressions consist of representations allied with separate processing systems and that those impressions can be affected by other activities involving those systems.     

Visuospatial working memory influences the interaction between space and time

How do representations of space inform our perception of time? In language, spatial vocabulary is frequently used to describe temporal concepts, and spatial information biases temporal perception even in non-verbal tasks. In contrast, temporal information typically exerts little, if any, influence on the perception of spatial extent. Here, we used spatial and temporal reproduction tasks, both with and without verbal and spatial dual tasks, to investigate the mechanism underlying the asymmetric relation between space and time. Specifically, we tested whether the asymmetric interference between spatial and temporal stimulus attributes arises from interference in verbal or visuospatial working memory. We found that loading visuospatial working memory, but not verbal working memory, eliminated the asymmetric pattern of interference. This suggests that the interference between spatial and temporal representations arises due to processing constraints in visuospatial working memory. These findings are discussed in terms of the load theory of attention and the relative automaticity with which spatial and temporal information is processed.     

Reward-priming impacts visual working memory maintenance: Evidence from human electrophysiology

Reward can benefit visual processing of reward-associated objects in a non-strategic way. Recent studies have suggested that such influence extends also to visual working memory (VWM) representations of reward-associated stimuli. However, it is not clear yet which mechanisms underlie the behavioural effects in VWM tasks: reward could directly impact on our ability to maintain representations in VWM or it could influence memory indirectly via priming of attentional selection. To distinguish between these alternatives we measured event-related potential indices of selective attention – the N2pc – and VWM maintenance – the CDA (contralateral delay activity) – while participants completed a VWM task. Results show that reward outcome in one trial caused similarly coloured targets to be strongly represented in VWM in subsequent trials, as expressed in a larger-amplitude CDA This was not preceded by a corresponding effect on attentional selection, in so far as our key manipulation had no impact on the N2pc. In a second experiment, we found that reward priming produced a behavioural benefit that emerged over time, suggesting that the representations of reward-associated items stored in VWM are more resistant to interference and decay. We conclude that when the task stresses VWM maintenance, it is at this representational level that reward will have impact.     

Interference in visual perception by verbal and spatial cognitive activity

In two experiments we tested the hypothesis that cognitive processing based on spatial imagery produces more deterioration of visual perception than cognitive processing based on verbal codes. So, we studied the effect on visual perception of two cognitive tasks, one of spatial imagery and the other a verbal task. In the first one, with 30 participants, we analyzed the mental load and ocular behaviors in both cognitive tasks. In the second experiment, with 29 participants, we studied the effect of both tasks on a visual search task, using a dual-task experimental paradigm. The verbal task presented higher mental load than the imagery task when both tasks were carried out with visual search task, and there was more deterioration in stimulus detection with the verbal task. We can conclude that: (1) cognitive tasks produce important deterioration in the capacities of visual search and identification of stimuli; (2) this deterioration has two components: (a) an inefficient search, associated with alterations of the gaze patterns while performing cognitive tasks, and (b) a general interference, nonspecific to spatial codes, in the process of identification of looked-at stimuli; (3) this cognitive interference is related to the mental load or effort required by the cognitive task.     

Talking and driving: applications of crossmodal action reveal a special role for spatial language

Talking reduces attention resulting in real-world crash risks to drivers that talk on a phone and drive. Driving is a behavior that is very demanding on spatial attention, suggesting potentially large interference by spatial codes in language. The current study investigated how different types of verbal codes influence visual attention during dual-task performance. In two experiments, participants performed a spatial or non-spatial verbal task while simultaneously performing a visual attention task. The results showed a larger decrement to visual attention performance when participants were concurrently engaged in a spatial verbal task. The results of the second experiment isolated this effect to the right cerebral hemisphere, consistent with a role for shared right parietal resources. These results are consistent with the idea that processing codes are an important component of coordinating talking and driving but generally inconsistent with a broad class of bottleneck approaches that describes dual-task decrements but treats component tasks as cognitively equivalent.     

Effects of concurrent spatial and verbal memory loads on serial position functions of laterally presented letter strings

In this study the influence of visuospatial and verbal memory loads on the identification of laterally presented letter strings was investigated. In the asymmetry task without concurrent loads, a clear right visual field advantage for letter identification was obtained. Hemisphere-specific effects due to concurrent loads were particularly observed when the difficulty of the load tasks was increased. The effects of visuospatial loads were found to be sex related, suggesting that under heavy load conditions mental rotation selectively overloads the processing capacity of the right hemisphere in males, while in females capacity limitations were observed in both hemispheres. Concurrent visuospatial loads produced more facilitation (or less interference) for letters in the outermost positions of each visual field than for letters in the innermost positions of each visual field. The results of the verbal memory load tasks revealed that an easy verbal load task facilitated performance which was particularly manifest for the right-most letter of both the left visual field and the right visual field. A difficult verbal memory load task interfered with recognition accuracy of letters which was most marked for the center letter in the right visual field. Letter position effects obtained in this study were interpreted in terms of various processing mechanisms influencing the serial position curves.     

Interference effects demonstrate distinct roles for visual and motor imagery during the mental representation of human action

Four experiments were completed to characterize the utilization of visual imagery and motor imagery during the mental representation of human action. In Experiment 1, movement time functions for a motor imagery human locomotion task conformed to a speed-accuracy trade-off similar to Fitts' Law, whereas those for a visual imagery object motion task did not. However, modality-specific interference effects in Experiment 2 demonstrate visual and motor imagery as cooperative processes when the action represented is tied to visual coordinates in space. Biomechanic-specific motor interference effects found in Experiment 3 suggest one basis for separation of processing channels within motor imagery. Finally, in Experiment 4 representations of motor actions were found to be generated using only visual imagery under certain circumstances: namely, when the imaginer represented the motor action of another individual while placed at an opposing viewpoint. These results suggest that the modality of representation recruited to generate images of human action is dependent on the dynamic relationship between the individual, movement, and environment.     

Memory-load interference in syntactic processing

Participants remembered a short set of words while reading syntactically complex sentences (object–extracted clefts) and syntactically simpler sentences (subject–extracted clefts) in a memory–load study. The study also manipulated whether the words in the set and the words in the sentence were of matched or unmatched types (common nouns vs. proper names). Performance in sentence comprehension was worse for complex sentences than for simpler sentences, and this effect was greater when the type of words in the memory load matched the type of words in the sentence. These results indicate that syntactic processing is not modular, instead suggesting that it relies on working memory resources that are used for other nonsyntactic processes. Further, the results indicate that similarity–based interference is an important constraint on information processing that can be overcome to some degree during language comprehension by using the coherence of language to construct integrated representations of meaning.     

Is recursion language-specific? Evidence of recursive mechanisms in the structure of intentional action

In their 2002 seminal paper Hauser, Chomsky and Fitch hypothesize that recursion is the only human-specific and language-specific mechanism of the faculty of language. While debate focused primarily on the meaning of recursion in the hypothesis and on the human-specific and syntax-specific character of recursion, the present work focuses on the claim that recursion is language-specific. We argue that there are recursive structures in the domain of motor intentionality by way of extending John R. Searle’s analysis of intentional action. We then discuss evidence from cognitive science and neuroscience supporting the claim that motor-intentional recursion is language-independent and suggest some explanatory hypotheses: (1) linguistic recursion is embodied in sensory-motor processing; (2) linguistic and motor-intentional recursions are distinct and mutually independent mechanisms. Finally, we propose some reflections about the epistemic status of HCF as presenting an empirically falsifiable hypothesis, and on the possibility of testing recursion in different cognitive domains.     

Implicit statistical learning in language processing: Word predictability is the key

Fundamental learning abilities related to the implicit encoding of sequential structure have been postulated to underlie language acquisition and processing. However, there is very little direct evidence to date supporting such a link between implicit statistical learning and language. In three experiments using novel methods of assessing implicit learning and language abilities, we show that sensitivity to sequential structure – as measured by improvements to immediate memory span for structurally-consistent input sequences – is significantly correlated with the ability to use knowledge of word predictability to aid speech perception under degraded listening conditions. Importantly, the association remained even after controlling for participant performance on other cognitive tasks, including short-term and working memory, intelligence, attention and inhibition, and vocabulary knowledge. Thus, the evidence suggests that implicit learning abilities are essential for acquiring long-term knowledge of the sequential structure of language – i.e., knowledge of word predictability – and that individual differences on such abilities impact speech perception in everyday situations. These findings provide a new theoretical rationale linking basic learning phenomena to specific aspects of spoken language processing in adults, and may furthermore indicate new fruitful directions for investigating both typical and atypical language development.