Investigating joint attention mechanisms through spoken human-robot interaction

Referential gaze during situated language production and comprehension is tightly coupled with the unfolding speech stream ( [Griffin, 2001] , [Meyer et al., 1998] and [Tanenhaus et al., 1995] ). In a shared environment, utterance comprehension may further be facilitated when the listener can exploit the speaker’s focus of (visual) attention to anticipate, ground, and disambiguate spoken references. To investigate the dynamics of such gaze-following and its influence on utterance comprehension in a controlled manner, we use a human–robot interaction setting. Specifically, we hypothesize that referential gaze is interpreted as a cue to the speaker’s referential intentions which facilitates or disrupts reference resolution. Moreover, the use of a dynamic and yet extremely controlled gaze cue enables us to shed light on the simultaneous and incremental integration of the unfolding speech and gaze movement.We report evidence from two eye-tracking experiments in which participants saw videos of a robot looking at and describing objects in a scene. The results reveal a quantified benefit-disruption spectrum of gaze on utterance comprehension and, further, show that gaze is used, even during the initial movement phase, to restrict the spatial domain of potential referents. These findings more broadly suggest that people treat artificial agents similar to human agents and, thus, validate such a setting for further explorations of joint attention mechanisms.     

Learning language from the input: why innate constraints can't explain noun compounding

Do the production and interpretation of patterns of plural forms in noun-noun compounds reveal the workings of innate constraints that govern morphological processing? The results of previous studies on compounding have been taken to support a number of important theoretical claims: first, that there are fundamental differences in the way that children and adults learn and process regular and irregular plurals, second, that these differences reflect formal constraints that govern the way the way regular and irregular plurals are processed in language, and third, that these constraints are unlikely to be the product of learning. In a series of seven experiments, we critically assess the evidence that is cited in support of these arguments. The results of our experiments provide little support for the idea that substantively different factors govern the patterns of acquisition, production and interpretation patterns of regular and irregular plural forms in compounds. Once frequency differences between regular and irregular plurals are accounted for, we find no evidence of any qualitative difference in the patterns of interpretation and production of regular and irregular plural nouns in compounds, in either adults or children. Accordingly, we suggest that the pattern of acquisition of both regular and irregular plurals in compounds is consistent with a simple account, in which children learn the conventions that govern plural compounding using evidence that is readily available in the distribution patterns of adult speech.     

Assessment of working memory components at 6years of age as predictors of reading achievements a year later

The ability of working memory skills (measured by tasks assessing all four working memory components), IQ, language, phonological awareness, literacy, rapid naming, and speed of processing at 6 years of age, before reading was taught, to predict reading abilities (decoding, reading comprehension, and reading time) a year later was examined in 97 children. Among all working memory components, phonological complex memory contributed most to predicting all three reading abilities. A capacity measure of phonological complex memory, based on passing a minimum threshold in those tasks, contributed to the explained variance of decoding and reading comprehension. Findings suggest that a minimal ability of phonological complex memory is necessary for children to attain a normal reading level. Adding assessment of phonological complex memory, before formal teaching of reading begins, to more common measures might better estimate children’s likelihood of future academic success.     

Learning to learn: From within-modality to cross-modality transfer during infancy

One critical aspect of learning is the ability to apply learned knowledge to new situations. This ability to transfer is often limited, and its development is not well understood. The current research investigated the development of transfer between 8 and 16 months of age. In Experiment 1, 8- and 16-month-olds (who were established to have a preference to the beginning of a visual sequence) were trained to attend to the end of a sequence. They were then tested on novel visual sequences. Results indicated transfer of learning, with both groups changing baseline preferences as a result of training. In Experiment 2, participants were trained to attend to the end of a visual sequence and were then tested on an auditory sequence. Unlike Experiment 1, only older participants exhibited transfer of learning by changing baseline preferences. These findings suggest that the generalization of learning becomes broader with development, with transfer across modalities developing later than transfer within a modality.     

Evidence for Implicit Learning in Syntactic Comprehension

This study provides evidence for implicit learning in syntactic comprehension. By reanalyzing data from a syntactic priming experiment (Thothathiri & Snedeker, 2008), we find that the error signal associated with a syntactic prime influences comprehenders' subsequent syntactic expectations. This follows directly from error‐based implicit learning accounts of syntactic priming, but it is unexpected under accounts that consider syntactic priming a consequence of temporary increases in base‐level activation. More generally, the results raise questions about the principles underlying the maintenance of implicit statistical knowledge relevant to language processing, and about possible functional motivations for syntactic priming.     

Modulation of N400 in chronic non-fluent aphasia using low frequency Repetitive Transcranial Magnetic Stimulation (rTMS)

Low frequency Repetitive Transcranial Magnetic Stimulation (rTMS) has previously been applied to language homologues in non-fluent populations of persons with aphasia yielding significant improvements in behavioral language function up to 43 months post stimulation. The present study aimed to investigate the electrophysiological correlates associated with the application of rTMS through measurement of the semantic based N400 Event-related brain potentials (ERP) component. Low frequency (1 Hz) rTMS was applied to the anterior portion of the homologue to Broca’s area (pars triangularis), for 20 min per day for 10 days, using a stereotactic neuronavigational system. Twelve non-fluent persons with aphasia, 2-6 years post stroke were stimulated. Six participants were randomly assigned to receive real stimulation and six participants were randomly assigned to receive a blind sham control condition. ERP measures were recorded at baseline, 1 week and 2 months subsequent to stimulation. The findings demonstrate treatment related changes observed in the stimulation group when compared to the placebo control group at 2 months post stimulation indicating neuromodulation of N400 as a result of rTMS. No treatment related changes were identified in the stimulation group, when compared to the sham group from baseline to 1 week post stimulation. The electrophysiological results represent the capacity of rTMS to modulate neural language networks and measures of lexical-semantic function in participants with non-fluent aphasia and suggest that time may be an important factor in brain reorganization subsequent to rTMS.     

Spectral information in nonspeech contexts influences children’s categorization of ambiguous speech sounds

For both adults and children, acoustic context plays an important role in speech perception. For adults, both speech and nonspeech acoustic contexts influence perception of subsequent speech items, consistent with the argument that effects of context are due to domain-general auditory processes. However, prior research examining the effects of context on children’s speech perception have focused on speech contexts; nonspeech contexts have not been explored previously. To better understand the developmental progression of children’s use of contexts in speech perception and the mechanisms underlying that development, we created a novel experimental paradigm testing 5-year-old children’s speech perception in several acoustic contexts. The results demonstrated that nonspeech context influences children’s speech perception, consistent with claims that context effects arise from general auditory system properties rather than speech-specific mechanisms. This supports theoretical accounts of language development suggesting that domain-general processes play a role across the lifespan.     

Border collie comprehends sentences containing a prepositional object,verb, and direct object

This study tested the ability of a Border Collie, Chaser, to learn the syntax and semantics of sentences consisting of three elements of grammar, a prepositional object, verb, and direct object. Understanding of the syntax of the sentences required that Chaser emit responses consistent with three elements of grammar sentences, such as to ball take Frisbee. Understanding of the semantics of the sentences required that Chaser respond correctly when the meanings of the sentences were changed by reversing positions of the prepositional object and the direct object in the sentence, such as to Frisbee take ball. Chaser's understanding of the sentences was tested in three different scenarios: (a) when multiple and familiar objects were used in the syntax command sentence, (b) when novel objects were used in the syntax command sentence (novel in the sense that objects had not been used during training), and (c) when vision of objects was not possible at the time the syntax command was verbalized. Findings were statistically significant in all three scenarios. Successful findings were attributed to Chaser's intensive training in her first three years of life. Analysis of the data revealed that Chaser's successful understanding of the syntax sentences required the processing and retention of two sound-object mappings (names-objects) into memory, along with simultaneous judgments concerning which object to take to the other – that is, working memory. These two types of cognitive abilities, memory storage and working memory, raise the bar in terms of our expectations of a dog's potential ability to understand verbal communications. We propose that Chaser's understanding of our three elements of grammar sentences represents a giant leap in her referential understanding of language.     

Language aptitude for pronunciation in advanced second language (L2) Learners: Behavioural predictors and neural substrates

Individual differences in second language (L2) aptitude have been assumed to depend upon a variety of cognitive and personality factors. Especially, the cognitive factor phonological working memory has been conceptualised as language learning device. However, strong associations between phonological working memory and L2 aptitude have been previously found in early-stage learners only, not in advanced learners. The current study aimed at investigating the behavioural and neurobiological predictors of advanced L2 learning. Our behavioural results showed that phonetic coding ability and empathy, but not phonological working memory, predict L2 pronunciation aptitude in advanced learners. Second, functional neuroimaging revealed this behavioural trait to be correlated with hemodynamic responses of the cerebral network of speech motor control and auditory-perceptual areas. We suggest that the acquisition of L2 pronunciation aptitude is a dynamic process, requiring a variety of neural resources at different processing stages over time.     

Processing of non-canonical word-order: A case-series on lesion-induced reorganized language and age-effects in typical development

Complex grammatical structures are mastered late in language acquisition. We studied age-effects on performance in object topicalization in 48 typically developing German-speaking participants (aged 8–30 years) and in five patients (children and adolescents) with lesion-induced atypical language representation. Production was tested by a sentence repetition task, comprehension by an acting out task. Three topicalized conditions with differing disambiguation (agreement, case, and case plus agreement) were contrasted with canonical control sentences.