Mastering the Art of Conversation in Middle Childhood – MACoMiC

The project relied on an interdisciplinary approach at the intersection of cognitive science, linguistics, computer science, and video analysis. It used corpora of child-caregiver conversations recorded in face-to-face settings and video calls, allowing the study of interaction in conditions closer to everyday life than standard laboratory tasks.

To analyze these data, the project combined human annotation with automatic tools from natural language processing, machine learning, and computer vision. These methods were used to study several dimensions of interaction, including speech acts, response relevance, communicative feedback, backchanneling, turn coordination, gaze, smiling, and laughter. This approach helped overcome part of the annotation bottleneck that typically limits the study of naturalistic interaction.

The project produced several important findings on children’s conversational development. It enabled the creation and analysis of original corpora for studying natural child-adult conversations, including multimodal interactions. It also led to the development of automatic tools to code or predict complex conversational behaviors.

The analyses show that several aspects of conversational coordination become relatively close to adult-like behavior by middle childhood, while others continue to develop more gradually. The project provided new results on the development of speech acts, communicative feedback, backchannel signals, multimodal coordination, gaze, and turn-taking. It also generated numerous scientific publications, several theses, and resources that will support future work on language development and human-machine interaction.

The project opens several promising directions. Scientifically, it provides a stronger basis for understanding how children learn to take part in complex social interaction. It also opens the way for future studies across other languages, sociocultural contexts, and atypical developmental trajectories.

Methodologically, the corpora and tools developed in the project can be reused to study natural conversation in greater detail. In the longer term, this work may also inform the design of better assessment tools for communicative development, as well as more child-adapted interactive systems in areas such as education, developmental health, and human-machine interaction.

Conversation is a ubiquitous and important activity in our lives. Cognitive scientists consider it as a hallmark of human cognition as it relies on a sophisticated ability for coordination and shared attention (Tomasello & Rakoczy, 2007; Laland & Seed, 2021). Prominent computer scientists have described it as the ultimate test for Artificial Intelligence (Turing, 1950). When conversational skills are not well developed, they can negatively impact our ability to learn from others and to maintain relationships (Murphy et al., 2014). Thus, the scientific study of how conversational skills develop in childhood and how these skills precisely relate to their underlying cognitive processes is of utmost importance to understand what makes human socio-cognition so special, to design better AI, and to allow more targeted and efficient clinical interventions (e.g., for individuals with autism). Yet, little is known about conversational development, largely because of the methodological limitations of traditional research methods. The current project leverages recent advances in machine learning to model children’s conversational development both at the behavioral and cognitive levels. The broad impact of this project is to develop a computational model of conversation that will not only advance our theoretical understanding but also serve as a computational foundation upon which several societal applications can be built.