Visual influences on interactive speech alignment

Speech alignment describes the unconscious tendency to produce speech that shares characteristics with perceived speech (eg Goldinger, 1998 Psychological Review 105 251-279). In the present study we evaluated whether seeing a talker enhances alignment over just hearing a talker. Pairs of participants performed an interactive search task which required them to repeatedly utter a series of keywords. Half of the pairs performed the task while hearing each other, while the other half could see and hear each other. Alignment was assessed by naive judges rating the similarity of interlocutors' keywords recorded before, during, and after the interactive task. Results showed that interlocutors aligned more when able to see one another suggesting that visual information enhances speech alignment.     

Coming to terms: quantifying the benefits of linguistic coordination

Sharing a public language facilitates particularly efficient forms of joint perception and action by giving interlocutors refined tools for directing attention and aligning conceptual models and action. We hypothesized that interlocutors who flexibly align their linguistic practices and converge on a shared language will improve their cooperative performance on joint tasks. To test this prediction, we employed a novel experimental design, in which pairs of participants cooperated linguistically to solve a perceptual task. We found that dyad members generally showed a high propensity to adapt to each other's linguistic practices. However, although general linguistic alignment did not have a positive effect on performance, the alignment of particular task-relevant vocabularies strongly correlated with collective performance. In other words, the more dyad members selectively aligned linguistic tools fit for the task, the better they performed. Our work thus uncovers the interplay between social dynamics and sensitivity to task affordances in successful cooperation.     

Interactive alignment as an allostatic process: A literature review

Research on interactive alignment encompasses various perspectives, including the recognition of linguistic coordination patterns, mental representation processes between interlocutors, and linguistic and behavioral convergence. In the absence of a literature review that presents the advances in the study of interactive alignment, this study identified 64 theoretical and empirical articles developed over 25 years (1996–2021) following the PRISMA methodology. Methodological and theoretical aspects derived from the analysis of the aforementioned reports are presented, showing how synchronization parameters interact, the associated neural bases, the impact of alignment on the development of joint actions, cognitive load, social interactions in general, and the prediction of language. The discussion and conclusions analyze the information presented and propose a comprehensive analysis from an allostatic perspective (predictive regulation through change) to advance the understanding of interactive alignment.     

Easier Said Than Done? Task Difficulty's Influence on Temporal Alignment,Semantic Similarity,and Complexity Matching Between Gestures and Speech

Gestures and speech are clearly synchronized in many ways. However, previous studies have shown that the semantic similarity between gestures and speech breaks down as people approach transitions in understanding. Explanations for these gesture–speech mismatches, which focus on gestures and speech expressing different cognitive strategies, have been criticized for disregarding gestures’ and speech's integration and synchronization. In the current study, we applied three different perspectives to investigate gesture–speech synchronization in an easy and a difficult task: temporal alignment, semantic similarity, and complexity matching. Participants engaged in a simple cognitive task and were assigned to either an easy or a difficult condition. We automatically measured pointing gestures, and we coded participant's speech, to determine the temporal alignment and semantic similarity between gestures and speech. Multifractal detrended fluctuation analysis was used to determine the extent of complexity matching between gestures and speech. We found that task difficulty indeed influenced gesture–speech synchronization in all three domains. We thereby extended the phenomenon of gesture–speech mismatches to difficult tasks in general. Furthermore, we investigated how temporal alignment, semantic similarity, and complexity matching were related in each condition, and how they predicted participants’ task performance. Our study illustrates how combining multiple perspectives, originating from different research areas (i.e., coordination dynamics, complexity science, cognitive psychology), provides novel understanding about cognitive concepts in general and about gesture–speech synchronization and task difficulty in particular.     

A Sequential Analysis of Depressive Behaviors Within Adolescent Peer Interactions

This study examined depressive and positive patterns of interactions among clinical and nonclinical adolescent peers within an interpersonal context. Ten clinical dyads with a depressed partner and 10 nonclinical dyads engaged in 16-min audiotaped conversations under positive and negative task instructions discussing positive and negative experiences. Conversations were unitized and coded according to depressive, aggressive, positive, and neutral behaviors. The clinical dyad-group demonstrated over two times more depressive behaviors than did the nonclinical dyad-group. Both dyad-groups exhibited increased depressive interactions during negative task instructions and increased positive interaction during positive task instructions. The loglinear approach to sequential analysis demonstrated significant overall bidirectional influence in the adolescents' dyadic interaction. Specifically, we found that the depressed adolescents' depressive behaviors decreased the likelihood of partners' aggressive behaviors, and increased the likelihood of partners' positive behaviors. Clinical and nonclinical dyad-groups showed reliable patterns of positive interaction sequences. Results demonstrate that depressive and positive behaviors are functional in adolescent dyadic interaction.     

Collective Contexts in Conversation: Grounding by Proxy

Anecdotal evidence suggests that participants in conversation can sometimes act as a coalition. This implies a level of conversational organization in which groups of individuals form a coherent unit. This paper investigates the implications of this phenomenon for psycholinguistic and semantic models of shared context in dialog. We present a corpus study of multiparty dialog which shows that, in certain circumstances, people with different levels of overt involvement in a conversation, that is, one responding and one not, can nonetheless access the same shared context. We argue that contemporary models of shared context need to be adapted to capture this situation. To address this, we propose “grounding by proxy,” in which one person can respond on behalf of another, as a simple mechanism by which shared context can accumulate for a coalition as a whole. We explore this hypothesis experimentally by investigating how people in a task‐oriented coalition respond when their shared context appears to be weakened. The results provide evidence that, by default, coalition members act on each other's behalf, and when this fails they work to compensate. We conclude that this points to the need for a new concept of collective grounding acts and a corresponding concept of collective contexts in psycholinguistic and semantic models of dialog.     

在家庭自由游戏和智力任务游戏中儿童与母亲的交往行为特征

本研究采用家庭录像观察的方法,考察了61名3－4岁儿童在家庭环境中的两个情境－－自由和带有智力任务倾向的拼图游戏情境中所表现出来的行为特征,结果发现：1儿童在两种游戏情境中与母亲交往时所表现出来的行为特征可以从独立性、顺从、言语交流和参与四个维度进行评价。2当儿童的游戏从自由游戏转为带有智力任务倾向的游戏时,儿童与母亲的交往行为方式发生明显变化,交流行为和积极情绪明显减少,而依赖行为和不与母亲交流的行为显著增加。3在上述两种游戏情境中,儿童与母亲的交往行为表现出个别差异,在自由游戏情境中,42.6%为独立型、32.8%为顺从型和24.6%为交流型;在拼图游戏情境中,54％为独立型、23％为顺从型和23％不顺从型。     

Good conversations: Grounding,convergence, and richness

We apply three communication theories (the collaborative theory of language use, communication accommodation theory, and media richness theory) to aspects of conversational structure (openings, closings) and communicative setting (audiovisual, written) in order to make predictions about how people will feel about conversational interactions. We distinguish between two sources of information that interlocutors use to evaluate conversations: conversational outcomes and subjective conversational experience. We propose that positive subjective conversational experience comes about when interlocutors reach common ground with one another and do so with the least required effort possible (collaborative theory of language use), converge on their language use and communicative behaviors (communication accommodation theory), and select the appropriate medium for their conversation to take place (media richness theory). This positive subjective conversational experience is an important component of good conversations.     

Spatial reasoning with external visualizations: what matters is what you see, not whether you interact

Three experiments examined the effects of interactive visualizations and spatial abilities on a task requiring participants to infer and draw cross sections of a three-dimensional (3D) object. The experiments manipulated whether participants could interactively control a virtual 3D visualization of the object while performing the task, and compared participants who were allowed interactive control of the visualization to those who were not allowed control. In Experiment 1, interactivity produced better performance than passive viewing, but the advantage of interactivity disappeared in Experiment 2 when visual input for the two conditions in a yoked design was equalized. In Experiments 2 and 3, differences in how interactive participants manipulated the visualization were large and related to performance. In Experiment 3, non-interactive participants who watched optimal movements of the display performed as well as interactive participants who manipulated the visualization effectively and better than interactive participants who manipulated the visualization ineffectively. Spatial ability made an independent contribution to performance on the spatial reasoning task, but did not predict patterns of interactive behavior. These experiments indicate that providing participants with active control of a computer visualization does not necessarily enhance task performance, whereas seeing the most task-relevant information does, and this is true regardless of whether the task-relevant information is obtained actively or passively.     

Gender and leadership aspiration: Interpersonal and collective elements of cooperative climate differentially influence women and men

Female leaders remain a minority. Because leadership aspiration is a predictor of advancement, understanding stimulating conditions is important. A neglected perspective is the impact of organizational climate. We propose that cooperative climate can engender individuals’ motivation to contribute to the organization through leadership, and that leadership aspiration of women and men is differentially sensitive to interpersonal and collective aspects of cooperative climate. We argue that women are more disposed toward relational self‐construal and men toward collective self‐construal, and hence women's leadership aspiration is more influenced by the interpersonal element of cooperative climate whereas men's leadership aspiration by the collective element of cooperative climate. Results of a survey of N = 404 employed men and women supported both hypotheses.     

Speech affordances: A structural take on how much we can do with our words

Individuals can do a broad variety of things with their words and enjoy different degrees of this capacity. What moderates this capacity? And in cases in which this capacity is unjustly disrupted, what is a good explanation for it? These are the questions I address here. I propose that speech capacity, understood as the capacity to do things with your words, is a structural property importantly dependent on individuals' position in a social structure. My account facilitates a non‐individualistic explanation of cases in which speech capacity is undermined due to speaker's perceived social identity, e.g. episodes of silencing. Instead of appealing to interlocutors' implicit bias against speaker's identity, a structural approach refers to the positions interlocutors occupy in the social structure and the discursive conventions operating upon those positions. I articulate my proposal drawing on the notion of affordances. Each position within a social structure is associated with its own range of speech affordances. Thus, speech capacity is a function of the probability distribution of speech affordances across positions in the structure.     

Difficulties in auditory organization as a cause of reading backwardness? An auditory neuroscience perspective

Over 30 years ago, it was suggested that difficulties in the ‘auditory organization’ of word forms in the mental lexicon might cause reading difficulties. It was proposed that children used parameters such as rhyme and alliteration to organize word forms in the mental lexicon by acoustic similarity, and that such organization was impaired in developmental dyslexia. This literature was based on an ‘oddity’ measure of children's sensitivity to rhyme (e.g. wood, book, good) and alliteration (e.g. sun, sock, rag). The ‘oddity’ task revealed that children with dyslexia were significantly poorer at identifying the ‘odd word out’ than younger children without reading difficulties. Here we apply a novel modelling approach drawn from auditory neuroscience to study the possible sensory basis of the auditory organization of rhyming and non‐rhyming words by children. We utilize a novel Spectral‐Amplitude Modulation Phase Hierarchy (S‐AMPH) approach to analysing the spectro‐temporal structure of rhyming and non‐rhyming words, aiming to illuminate the potential acoustic cues used by children as a basis for phonological organization. The S‐AMPH model assumes that speech encoding depends on neuronal oscillatory entrainment to the amplitude modulation (AM) hierarchy in speech. Our results suggest that phonological similarity between rhyming words in the oddity task depends crucially on slow (delta band) modulations in the speech envelope. Contrary to linguistic assumptions, therefore, auditory organization by children may not depend on phonemic information for this task. Linguistically, it is assumed that ‘book’ does not rhyme with ‘wood’ and ‘good’ because the final phoneme differs. However, our auditory analysis suggests that the acoustic cues to this phonological dissimilarity depend primarily on the slower amplitude modulations in the speech envelope, thought to carry prosodic information. Therefore, the oddity task may help in detecting reading difficulties because phonological similarity judgements about rhyme reflect sensitivity to slow amplitude modulation patterns. Slower amplitude modulations are known to be detected less efficiently by children with dyslexia.     

Exploring emotions in dialog between health provider,parent and child. An observational study in pediatric primary care

IntroductionThis paper is part of the theoretical thread of health psychology and it presents a study aimed to recognize information useful to implement psychological interventions finalized to personalize medical treatments and engage users in pediatrics.ObjectiveIn this paper is presented an observational study aimed to explore doctor-patient dialog about worries in pediatric primary care; it has been carried out identifying dialogical interaction patterns, or typical dialog between health providers, parent and child.MethodWe documented conversations in 265 visits; we audio-recordered, transcribed and analyzed them with Verona Coding Definitions of Emotional Sequences. Dialogs between participants were analyzed one by one. Frequencies of emotions’ signals (cues/concerns) and responses were analyzed through Redundancy Analysis, aimed to establish a quantitative relationship between these pair of groups of variables considering the asymmetrical relationship between them.ResultsSix “Dialogical Interaction Patterns” were obtained by interpreting these relationships; they show dialogs mainly aimed at obtaining information useful for diagnosis and treatment, with a limited exploration of worries or issues related to the condition of the child. Pediatric conversations seem to be characterized by a very high attention to cognitive aspects of medical questions with a poor consideration of emotions as useful information to medical practice.ConclusionsThese dialogs seem to ensure the rapidity and the efficiency of medical visits. Nevertheless, it could be useful to implement psychological interventions to achieve an enrichment of the dialog between participants, helping them to recognize users’ emotions as useful to define shared medical strategies.     

Poetic Confluence: A Sociological Analysis of an Enigmatic Moment

This paper examines a form of interpersonal relationality that takes the form of a speech event in which one participant produces a spoken turn that exhibits a poetic relationship to a co-participant’s unspoken thoughts or unarticulated mental imagery. This examined in relation to an earlier analysis of a speech error during therapy which appeared to reflect some form of telepathic communication between patient and analyst. Drawing from sociological studies of the organization of everyday social interaction, I sketch some ways in which a sociological approach can contribute to psychoanalytic reflections on telepathic experiences between patient and analysts.     

Effects of group goal content on group processes,collective efficacy,and satisfaction

We examined the efficacy of groups possessing learning as opposed to performance goals on an interactive group task. As such, we predicted that the possession of learning goals focuses groups more on strategic processes than the possession of performance or do‐best goals. We further hypothesized that collective efficacy beliefs are most strongly correlated with performance for groups that possess performance goals because performance goals direct attention to task outcomes. Results indicated that groups assigned learning goals discussed more strategic information and reported greater satisfaction with performance. In addition, the relationship between task performance and collective efficacy was stronger for performance‐goal groups than learning‐goal groups. Overall, results suggested that causing groups to attend to processes required for effective group performance influences how groups regulate behavior and interpret feedback.     

Coordinated interpersonal timing in the conversations of children who stutter and their mothers and fathers

Coordinated interpersonal timing (CIT) is a measure of "conversational congruence," or "attunement," and refers to the degree to which the temporal aspects of the vocal behaviors of co-conversationalists are correlated over the course of a conversation [Jasnow, M., & Feldstein, S. (1986). Adult-like temporal characteristics of mother-infant vocal interaction. Child Development, 57, 754-761]. In the present study, CIT was examined in a group of children who stutter (CWS), and a matched group of nonstuttering children (CWDNS; children who do not stutter), during conversations with either their mother or father recorded in two separate sessions (i.e., mother-child, father-child). Separate audio signals for both the child and parent (mother or father) were analyzed using AVTA software, which allowed for the quantification of sound and silence patterns in simultaneous speech. Squared cross-correlations (i.e., coefficients of CIT) for the durations of five vocal behavior states were obtained for each subject, through time series regression analysis using lag procedures. Vocal state behaviors within conversational turns included: vocalization, pauses, turn switching pauses, and interruptive and noninterruptive simultaneous speech. Results indicated that CWS and their parents showed mutual influence (i.e., CIT in both directions, child to parent and parent to child, or bi-directional influence) for more vocal state behaviors than did CWDNS and their parents. In addition, the CWS exhibited CIT with their parents for the durations of more vocal state behaviors than did the CWDNS (i.e., unidirectional influence). Findings suggest that children who stutter may be more easily influenced by the subtle timing aspects of conversation. Taken further, some of these children may perceive conversations with their parents as either challenging or difficult because of an element of unpredictability brought into conversations by the production of stuttering, the social skills of the child, and the nature of the parent-child relationship. Consequently, these children may be engaging in more pervasive coordination of the temporal characteristics of their speech to those of their conversational partner, as a mechanism by which to more effectively manage verbal interaction. EDUCATIONAL OBJECTIVES: After reading this paper, the learner will be able to: (1) describe the phenomenon of coordinated interpersonal timing (CIT); (2) summarize research findings in CIT as they apply to the verbal interactions of children and their parents; (3) summarize research findings in parent-stuttering child interaction, especially those related to the temporal aspects of both parent and child conversational speech, and (4) discuss the applicability of the findings from the present study to the treatment of childhood stuttering.     

Alignment to visual speech information

Speech alignment is the tendency for interlocutors to unconsciously imitate one another’s speaking style. Alignment also occurs when a talker is asked to shadow recorded words (e.g., Shockley, Sabadini, & Fowler, 2004). In two experiments, we examined whether alignment could be induced with visual (lipread) speech and with auditory speech. In Experiment 1, we asked subjects to lipread and shadow out loud a model silently uttering words. The results indicate that shadowed utterances sounded more similar to the model’s utterances than did subjects’ nonshadowed read utterances. This suggests that speech alignment can be based on visual speech. In Experiment 2, we tested whether raters could perceive alignment across modalities. Raters were asked to judge the relative similarity between a model’s visual (silent video) utterance and subjects’ audio utterances. The subjects’ shadowed utterances were again judged as more similar to the model’s than were read utterances, suggesting that raters are sensitive to cross-modal similarity between aligned words.     

Goals as a mediator of gender differences in high-affiliation dyadic conversations

The present study examined whether gender differences in affiliative aspects (collaboration and cooperation) of dyadic conversations occur because girls are more oriented than boys toward goals focused on others. Preadolescents (11-13 years old; 51 boys, 53 girls) worked with a same- or an other-gender peer on a 4-week-long creative-writing task at school. Dyadic conversations and goals were assessed twice. High-affiliation conversations and mutual-participation goals were more prevalent in female than in male and mixed-gender dyads. Mutual-participation goals mediated gender differences in high-affiliation conversations. Control and task-performance goals did not differ by dyad gender. In mixed-gender dyads, conversation strategies and goals did not differ by gender. Implications of goals for understanding gender differences and similarities in conversations are discussed.     

A dynamic view of cultural influence: A review

Static models of culture's influence have given way to a dynamic view, which identifies not only differences across cultures in people's judgments and decisions, but also the situations and conditions in which these differences do or do not appear. Theory and evidence developed from a cognitive psychological perspective underlie this dynamic approach, including research emerging from the “dynamic constructivist” and “situated cognition” models. In the present review, we focus on findings that confirm the utility of this cognitively oriented approach, and briefly discuss the advantages and complementary nature of the “social collective” and neuroscience approaches to understanding culture.