Building phrases in language production: An MEG study of simple composition

Although research on language production has developed detailed maps of the brain basis of single word production in both time and space, little is known about the spatiotemporal dynamics of the processes that combine individual words into larger representations during production. Studying composition in production is challenging due to difficulties both in controlling produced utterances and in measuring the associated brain responses. Here, we circumvent both problems using a minimal composition paradigm combined with the high temporal resolution of magnetoencephalography (MEG). With MEG, we measured the planning stages of simple adjective–noun phrases (‘red tree’), matched list controls (‘red, blue’), and individual nouns (‘tree’) and adjectives (‘red’), with results indicating combinatorial processing in the ventro-medial prefrontal cortex (vmPFC) and left anterior temporal lobe (LATL), two regions previously implicated for the comprehension of similar phrases. These effects began relatively quickly (∼180 ms) after the presentation of a production prompt, suggesting that combination commences with initial lexical access. Further, while in comprehension, vmPFC effects have followed LATL effects, in this production paradigm vmPFC effects occurred mostly in parallel with LATL effects, suggesting that a late process in comprehension is an early process in production. Thus, our results provide a novel neural bridge between psycholinguistic models of comprehension and production that posit functionally similar combinatorial mechanisms operating in reversed order.     

Social eye gaze modulates processing of speech and co-speech gesture

In human face-to-face communication, language comprehension is a multi-modal, situated activity. However, little is known about how we combine information from different modalities during comprehension, and how perceived communicative intentions, often signaled through visual signals, influence this process. We explored this question by simulating a multi-party communication context in which a speaker alternated her gaze between two recipients. Participants viewed speech-only or speech + gesture object-related messages when being addressed (direct gaze) or unaddressed (gaze averted to other participant). They were then asked to choose which of two object images matched the speaker’s preceding message. Unaddressed recipients responded significantly more slowly than addressees for speech-only utterances. However, perceiving the same speech accompanied by gestures sped unaddressed recipients up to a level identical to that of addressees. That is, when unaddressed recipients’ speech processing suffers, gestures can enhance the comprehension of a speaker’s message. We discuss our findings with respect to two hypotheses attempting to account for how social eye gaze may modulate multi-modal language comprehension.     

The influence of speaker gaze on listener comprehension: Contrasting visual versus intentional accounts

Previous research has shown that listeners follow speaker gaze to mentioned objects in a shared environment to ground referring expressions, both for human and robot speakers. What is less clear is whether the benefit of speaker gaze is due to the inference of referential intentions (Staudte and Crocker, 2011) or simply the (reflexive) shifts in visual attention. That is, is gaze special in how it affects simultaneous utterance comprehension? In four eye-tracking studies we directly contrast speech-aligned speaker gaze of a virtual agent with a non-gaze visual cue (arrow). Our findings show that both cues similarly direct listeners’ attention and that listeners can benefit in utterance comprehension from both cues. Only when they are similarly precise, however, does this equality extend to incongruent cueing sequences: that is, even when the cue sequence does not match the concurrent sequence of spoken referents can listeners benefit from gaze as well as arrows. The results suggest that listeners are able to learn a counter-predictive mapping of both cues to the sequence of referents. Thus, gaze and arrows can in principle be applied with equal flexibility and efficiency during language comprehension.     

Adult language use and infant comprehension of English: Associations with encoding and generalization across cues at 20 months

Adult-provided language shapes event memory in children who are preverbal and in those who are able to discuss the past using language. The research conducted to date, however, has not yet established whether infant language comprehension abilities moderate the extent to which preverbal infants benefit from adult-provided supportive language. The present study was conducted to address this question by examining immediate imitation and 1-week delayed generalization across cues in 20-month-old infants as a function of (a) variability in adult-provided linguistic support at encoding and test, (b) infant language comprehension abilities, and (c) their interaction. The provision of supportive adult language at encoding and test was associated with delayed generalization across cues although supportive adult language at encoding did not influence performance at immediate imitation. Infant language comprehension abilities were associated with performance at immediate imitation and delayed generalization across cues. In addition, infant language comprehension abilities moderated the extent to which infants benefited from adult-provided supportive language at encoding and test. The findings contribute to the literature by demonstrating that adult language use and infant language comprehension are independently and differentially associated with immediate imitation and 1-week delayed generalization across cues but also serve to jointly structure event memory in the second year of life.     

Happy babies,chatty toddlers: Infant positive affect facilitates early expressive,but not receptive language

Eighty-three mother–infant dyads participated in this study. Positive affect (PA) broadly, along with fine-grained aspects of PA, was measured at 10 months of age. Language was measured at 14 months. Infant PA predicted expressive, but not receptive, language. The implications of these findings are discussed.     

Emergence,Story, and the Challenge of Positive Scenarios

This article is divided into three parts. The first part describes the paradox of positive scenarios: they are much harder to write than negative scenarios. The second part describes the logic of emergent systems whose properties are neither reducible to nor predictable from the properties of their component parts. The third part shows how the concept of emergence can be used to lend plausibility to those optimistic scenarios that might otherwise be rejected as mere wishful thinking.     

Perspective-taking in dialogue as self-organization under social constraints

We present a dynamical systems account of how simple social information influences perspective-taking. Our account is motivated by the notion that perspective-taking may obey common dynamic principles with perceptuomotor coupling. We turn to the prominent HKB dynamical model of motor coordination, drawing from basic principles of self-organization to describe how conversational perspective-taking unfolds in a low-dimensional attractor landscape. We begin by simulating experimental data taken from a simple instruction-following task, in which participants have different expectations about their interaction partner. By treating belief states as different values of a control parameter, we show that data generated by a basic dynamical process fits overall egocentric and other-centric response distributions, the time required for participants to enact a response on a trial-by-trial basis, and the action dynamics exhibited in individual trials. We end by discussing the theoretical significance of dynamics in dialog, arguing that high-level coordination such as perspective-taking may obey similar dynamics as perceptuomotor coordination, pointing to common principles of adaptivity and flexibility during dialog.     

Talking as doing: Language forms and public language

I discuss language forms as the primary means that language communities provide to enable public language use. As such, they are adapted to public use most notably in being linguistically significant vocal tract actions, not the categories in the mind as proposed in phonological theories. Their primary function is to serve as vehicles for production of syntactically structured sequences of words. However, more than that, phonological actions themselves do work in public language use. In particular, they foster interpersonal coordination in social activities. An intriguing property of language forms that likely reflects their emergence in social communicative activities is that phonological forms that should be meaningless (in order to serve their role in the openness of language at the level of the lexicon) are not wholly meaningless. In fact, the form-meaning “rift” is bridged bidirectionally: The smallest language forms are meaningful, and the meanings of lexical language forms generally inhere, in part, in their embodiment by understanders.     

What Does Children's Spatial Language Reveal About Spatial Concepts? Evidence From the Use of Containment Expressions

Children's overextensions of spatial language are often taken to reveal spatial biases. However, it is unclear whether extension patterns should be attributed to children's overly general spatial concepts or to a narrower notion of conceptual similarity allowing metaphor‐like extensions. We describe a previously unnoticed extension of spatial expressions and use a novel method to determine its origins. English‐ and Greek‐speaking 4‐ and 5‐year‐olds used containment expressions (e.g., English into, Greek mesa) for events where an object moved into another object but extended such expressions to events where the object moved behind or under another object. The pattern emerged in adult speakers of both languages and also in speakers of 10 additional languages. We conclude that learners do not have an overly general concept of Containment. Nevertheless, children (and adults) perceive similarities across Containment and other types of spatial scenes, even when these similarities are obscured by the conventional forms of the language.