Effects of Gestures on Older Adults' Learning from Video‐based Models

This study investigated whether the positive effects of gestures on learning by decreasing working memory load, found in children and young adults, also apply to older adults, who might especially benefit from gestures given memory deficits associated with aging. Participants learned a problem‐solving skill by observing a video‐based modeling example, with the human model using gesture cues, with a symbolic cue, or without cues. It was expected that gesture compared with symbolic or no cues (i) improves learning and transfer performance, (ii) more in complex than simple problems, and (iii) especially in older adults. Although older adults' learning outcomes were lower overall than that of children and young adults, the results only revealed a time‐on‐task advantage of gesture over no cues in the learning phase for the older adults. In conclusion, the present study did not provide strong support for the effectiveness of gestures on learning from video‐based modeling example. Copyright © 2014 John Wiley & Sons, Ltd.     

追踪手势对视空间学习的增强作用

本研究采用地图学习任务范式考察了追踪手势对视空间学习的影响和增强, 并试图从追踪手势的角度寻找能够有效促进操作者视空间学习的方法。实验1探究了产生追踪手势对视空间学习的增强作用; 实验2采用遮挡范式考察了产生追踪手势增强视空间学习的作用机制, 发现产生追踪手势所提供的视觉信息和感觉运动信息共同作用于视空间学习过程; 实验3从具身认知的视角探索了基于追踪手势的视空间学习增强方式, 发现以自我为参照产生追踪手势可以更好地增强个体的视空间学习。本研究结果很好地支持并补充了手势的具身认知理论。     

Doing gesture promotes learning a mental transformation task better than seeing gesture

Performing action has been found to have a greater impact on learning than observing action. Here we ask whether a particular type of action – the gestures that accompany talk – affect learning in a comparable way. We gave 158 6‐year‐old children instruction in a mental transformation task. Half the children were asked to produce a Move gesture relevant to the task; half were asked to produce a Point gesture. The children also observed the experimenter producing either a Move or Point gesture. Children who produced a Move gesture improved more than children who observed the Move gesture. Neither producing nor observing the Point gesture facilitated learning. Doing gesture promotes learning better than seeing gesture, as long as the gesture conveys information that could help solve the task.     

Gesturing during mental problem solving reduces eye movements,especially for individuals with lower visual working memory capacity

Non-communicative hand gestures have been found to benefit problem-solving performance. These gestures seem to compensate for limited internal cognitive capacities, such as visual working memory capacity. Yet, it is not clear how gestures might perform this cognitive function. One hypothesis is that gesturing is a means to spatially index mental simulations, thereby reducing the need for visually projecting the mental simulation onto the visual presentation of the task. If that hypothesis is correct, less eye movements should be made when participants gesture during problem solving than when they do not gesture. We therefore used mobile eye tracking to investigate the effect of co-thought gesturing and visual working memory capacity on eye movements during mental solving of the Tower of Hanoi problem. Results revealed that gesturing indeed reduced the number of eye movements (lower saccade counts), especially for participants with a relatively lower visual working memory capacity. Subsequent problem-solving performance was not affected by having (not) gestured during the mental solving phase. The current findings suggest that our understanding of gestures in problem solving could be improved by taking into account eye movements during gesturing.     

Individual differences in the gesture effect on working memory

Co-speech gestures have been shown to interact with working memory (WM). However, no study has investigated whether there are individual differences in the effect of gestures on WM. Combining a novel gesture/no-gesture task and an operation span task, we examined the differences in WM accuracy between individuals who gestured and individuals who did not gesture in relation to their WM capacity. Our results showed individual differences in the gesture effect on WM. Specifically, only individuals with low WM capacity showed a reduced WM accuracy when they did not gesture. Individuals with low WM capacity who did gesture, as well as high-capacity individuals (irrespective of whether they gestured or not), did not show the effect. Our findings show that the interaction between co-speech gestures and WM is affected by an individual’s WM load.     

Unhindered Spatial Processing During Route Memorization is Required to Maximize Both Spatial and Verbal Route Knowledge

To successfully remember a route in a physical environment, individuals are believed to process and store both verbal and spatial information of that route. The present study tested whether both spatial and verbal contents are necessary to form an effective route memory. For that purpose, route learning was performed in three concurrent task conditions (spatial, verbal and control) and appraised at two moments in time, via three route memory tests (spatial, verbal and spatial–verbal). Results showed that route memory generally improved across time and that spatial information was remembered better than verbal information. The concurrent spatial condition resulted in lower retention scores of both spatial and verbal route knowledge. These results suggest that effective spatial processing forms a scaffold without which long‐term retention lacks detail of both spatial and verbal route knowledge. It is discussed how these findings add to understanding spatial memory of routes. Copyright © 2013 John Wiley & Sons, Ltd.     

Are depictive gestures like pictures? commonalities and differences in semantic processing

Conversation is multi-modal, involving both talk and gesture. Does understanding depictive gestures engage processes similar to those recruited in the comprehension of drawings or photographs? Event-related brain potentials (ERPs) were recorded from neurotypical adults as they viewed spontaneously produced depictive gestures preceded by congruent and incongruent contexts. Gestures were presented either dynamically in short, soundless video-clips, or statically as freeze frames extracted from gesture videos. In a separate ERP experiment, the same participants viewed related or unrelated pairs of photographs depicting common real-world objects. Both object photos and gesture stimuli elicited less negative ERPs from 400 to 600 ms post-stimulus when preceded by matching versus mismatching contexts (dN450). Object photos and static gesture stills also elicited less negative ERPs between 300 and 400 ms post-stimulus (dN300). Findings demonstrate commonalities between the conceptual integration processes underlying the interpretation of iconic gestures and other types of image-based representations of the visual world.     

Co‐thought gestures in children's mental problem solving: Prevalence and effects on subsequent performance

Co‐thought gestures are understudied as compared to co‐speech gestures yet, may provide insight into cognitive functions of gestures that are independent of speech processes. A recent study with adults showed that co‐thought gesticulation occurred spontaneously during mental preparation of problem solving. Moreover, co‐thought gesturing (either spontaneous or instructed) during mental preparation was effective for subsequent solving of the Tower of Hanoi under conditions of high cognitive load (i.e., when visual working memory capacity was limited and when the task was more difficult). In this preregistered study ( https://osf.io/dreks/ ), we investigated whether co‐thought gestures would also spontaneously occur and would aid problem‐solving processes in children (N = 74; 8–12 years old) under high load conditions. Although children also spontaneously used co‐thought gestures during mental problem solving, this did not aid their subsequent performance when physically solving the problem. If these null results are on track, co‐thought gesture effects may be different in adults and children.     

Co-speech iconic gestures and visuo-spatial working memory

Three experiments tested the role of verbal versus visuo-spatial working memory in the comprehension of co-speech iconic gestures. In Experiment 1, participants viewed congruent discourse primes in which the speaker's gestures matched the information conveyed by his speech, and incongruent ones in which the semantic content of the speaker's gestures diverged from that in his speech. Discourse primes were followed by picture probes that participants judged as being either related or unrelated to the preceding clip. Performance on this picture probe classification task was faster and more accurate after congruent than incongruent discourse primes. The effect of discourse congruency on response times was linearly related to measures of visuo-spatial, but not verbal, working memory capacity, as participants with greater visuo-spatial WM capacity benefited more from congruent gestures. In Experiments 2 and 3, participants performed the same picture probe classification task under conditions of high and low loads on concurrent visuo-spatial (Experiment 2) and verbal (Experiment 3) memory tasks. Effects of discourse congruency and verbal WM load were additive, while effects of discourse congruency and visuo-spatial WM load were interactive. Results suggest that congruent co-speech gestures facilitate multi-modal language comprehension, and indicate an important role for visuo-spatial WM in these speech–gesture integration processes.     

儿童手势及其与学习的关系

手势是在交流或认知过程中产生的、不直接作用于物体的手部运动, 具有具体性和抽象性。其分类主要从手势的来源、手势的内容、手势的意图及手势和言语的匹配性等角度进行划分。不同类型手势在出现时间及发展趋势上存在差异。手势在儿童词汇学习、言语表达、数学问题解决、空间学习及记忆等方面起促进作用, 但对言语理解的影响未得出一致结论。未来可关注不同类型手势与儿童认知发展的关系及对比不同来源手势在各学习领域的优势情况。     

Working memory and phonological awareness as predictors of progress towards early learning goals at school entry

This study investigates whether working memory skills of children are related to teacher ratings of their progress towards learning goals at the time of school entry, at 4 or 5 years of age. A sample of 194 children was tested on measures of working memory, phonological awareness, and non‐verbal ability, in addition to the school‐based baseline assessments in the areas of reading, writing, mathematics, speaking and listening, and personal and social development. Various aspects of cognitive functioning formed unique associations with baseline assessments; for example complex memory span with rated writing skills, phonological short‐term memory with both reading and speaking and listening skills, and sentence repetition scores with both mathematics and personal and social skills. Rated reading skills were also uniquely associated with phonological awareness scores. The findings indicate that the capacity to store and process material over short periods of time, referred to as working memory, and also the awareness of phonological structure, may play a crucial role in key learning areas for children at the beginning of formal education.     

Representations in mental imagery and working memory: Evidence from different types of visual masks

Although few studies have systematically investigated the relationship between visual mental imagery and visual working memory, work on the effects of passive visual interference has generally demonstrated a dissociation between the two functions. In four experiments, we investigated a possible commonality between the two functions: We asked whether both rely on depictive representations. Participants judged the visual properties of letters using visual mental images or pictures of unfamiliar letters stored in short-term memory. Participants performed both tasks with two different types of interference: sequences of unstructured visual masks (consisting of randomly changing white and black dots) or sequences of structured visual masks (consisting of fragments of letters). The structured visual noise contained elements of depictive representations (i.e., shape fragments arrayed in space), and hence should interfere with stored depictive representations; the unstructured visual noise did not contain such elements, and thus should not interfere as much with such stored representations. Participants did in fact make more errors in both tasks with sequences of structured visual masks. Various controls converged in demonstrating that in both tasks participants used representations that depicted the shapes of the letters. These findings not only constrain theories of visual mental imagery and visual working memory, but also have direct implications for why some studies have failed to find that dynamic visual noise interferes with visual working memory.     

The role of gestures in spatial working memory and speech

Co-speech gestures traditionally have been considered communicative, but they may also serve other functions. For example, hand-arm movements seem to facilitate both spatial working memory and speech production. It has been proposed that gestures facilitate speech indirectly by sustaining spatial representations in working memory. Alternatively, gestures may affect speech production directly by activating embodied semantic representations involved in lexical search. Consistent with the first hypothesis, we found participants gestured more when describing visual objects from memory and when describing objects that were difficult to remember and encode verbally. However, they also gestured when describing a visually accessible object, and gesture restriction produced dysfluent speech even when spatial memory was untaxed, suggesting that gestures can directly affect both spatial memory and lexical retrieval.     

Early Gesture Provides a Helping Hand to Spoken Vocabulary Development for Children with Autism,Down Syndrome,and Typical Development

Typically developing (TD) children refer to objects uniquely in gesture (e.g., point at a cat) before they produce verbal labels for these objects (“cat”). The onset of such gestures predicts the onset of similar spoken words, showing a strong positive relation between early gestures and early words. We asked whether gesture plays the same door-opening role in word learning for children with autism spectrum disorder (ASD) and Down syndrome (DS), who show delayed vocabulary development and who differ in the strength of gesture production. To answer this question, we observed 23 18-month-old TD children, 23 30-month-old children with ASD, and 23 30-month-old children with DS 5 times over a year during parent–child interactions. Children in all 3 groups initially expressed a greater proportion of referents uniquely in gesture than in speech. Many of these unique gestures subsequently entered children’s spoken vocabularies within a year—a pattern that was slightly less robust for children with DS, whose word production was the most markedly delayed. These results indicate that gesture is as fundamental to vocabulary development for children with developmental disorders as it is for TD children.     

Verbal working memory predicts co-speech gesture: Evidence from individual differences

Gesture facilitates language production, but there is debate surrounding its exact role. It has been argued that gestures lighten the load on verbal working memory (VWM; Goldin-Meadow, Nusbaum, Kelly, & Wagner, 2001), but gestures have also been argued to aid in lexical retrieval (Krauss, 1998). In the current study, 50 speakers completed an individual differences battery that included measures of VWM and lexical retrieval. To elicit gesture, each speaker described short cartoon clips immediately after viewing. Measures of lexical retrieval did not predict spontaneous gesture rates, but lower VWM was associated with higher gesture rates, suggesting that gestures can facilitate language production by supporting VWM when resources are taxed. These data also suggest that individual variability in the propensity to gesture is partly linked to cognitive capacities.     

Gesture Helps,Only If You Need It: Inhibiting Gesture Reduces Tip-of-the-Tongue Resolution for Those With Weak Short-Term Memory

People frequently gesture when a word is on the tip of their tongue (TOT), yet research is mixed as to whether and why gesture aids lexical retrieval. We tested three accounts: the lexical retrieval hypothesis, which predicts that semantically related gestures facilitate successful lexical retrieval; the cognitive load account, which predicts that matching gestures facilitate lexical retrieval only when retrieval is hard, as in the case of a TOT; and the motor movement account, which predicts that any motor movements should support lexical retrieval. In Experiment 1 (a between-subjects study; N = 90), gesture inhibition, but not neck inhibition, affected TOT resolution but not overall lexical retrieval; participants in the gesture-inhibited condition resolved fewer TOTs than participants who were allowed to gesture. When participants could gesture, they produced more representational gestures during resolved than unresolved TOTs, a pattern not observed for meaningless motor movements (e.g., beats). However, the effect of gesture inhibition on TOT resolution was not uniform; some participants resolved many TOTs, while others struggled. In Experiment 2 (a within-subjects study; N = 34), the effect of gesture inhibition was traced to individual differences in verbal, not spatial short-term memory (STM) span; those with weaker verbal STM resolved fewer TOTs when unable to gesture. This relationship between verbal STM and TOT resolution was not observed when participants were allowed to gesture. Taken together, these results fit the cognitive load account; when lexical retrieval is hard, gesture effectively reduces the cognitive load of TOT resolution for those who find the task especially taxing.     

Reduced Verbal Memory Retention is Unrelated to Sleep Disturbance During Pregnancy

This study investigated episodic and procedural memory retention in early and late pregnancy and whether memory retention was related to sleep disruption. Twenty‐six women in the third trimester of pregnancy, 20 women in the first trimester of pregnancy, and 24 non‐pregnant controls were administered a battery of verbal and visual episodic memory tasks and two procedural memory tasks before undergoing an overnight sleep study. Memory retention was assessed the following morning. Results indicated that as compared with controls, both pregnant groups had reduced retention in verbal episodic memory but were unimpaired on visual and procedural memory tasks. The pregnant women also demonstrated significant disruption of sleep patterns. Reduced verbal memory retention during pregnancy was not attributable to any measure of sleep; however, small correlations between some indices of sleep and memory do not allow full dismissal of the sleep‐dependent memory consolidation hypothesis.     

Division of the articulatory loop according to sensory modality using double dissociation

According to Wilson and Fox (2007), working memory for gestures has the same characteristics as the phonological loop. The purpose of our research was to determine whether there is a common articulatory loop for verbal and gestural learning. We carried out two double dissociation experiments. The first involved 84 participants who had to reproduce a series of three gestures under three conditions: control, gestural interference (repeated gestures) and verbal interference (repeated “blah blah”). A significant difference in performance was observed; gestural interference resulted in the weakest performance, while there was no difference between the verbal interference condition and the control group. The second experiment, with 30 participants, involved the memorisation of letters and digits; performance was significantly affected by verbal interference but there was no difference between the gestural interference condition and the control group. The consequences of the dissociations are discussed in relation to Baddeley's (2000) model.