FRAL - Programme franco-allemand en Sciences humaines et sociales

Spoken lAnguage in motions: Learning and Adaptation of speech coMMunication in the context of BOdy motions – Salammbo

Submission summary

In everyday situations, we often speak while moving (e.g., walking together, cooking, knitting), and we move while speaking (e.g. head, arms and hands, posture). Breathing mediates between speech and motion, and is a source for these activities: It provides the brain and muscles with the required oxygen, and the expiratory airflow is needed for the production of sound. Breathing is also a pacemaker for the speech flow: inhalation pauses are coordinated with prosody and syntax.

Following recent theories on situated and embodied cognition and language, Salammbo’s originality will be to consider limb motion as a common context for spoken language, and breathing as a mediator between limbs and spoken language. It will adopt an interdisciplinary approach integrating linguistics, movement science and psychology. The first aim is to create a novel multimodal corpus, with simultaneous recordings of limb motion, respiratory, articulatory and acoustic data using advanced technology. A cross-linguistic longitudinal approach will be adopted. Native speakers of French and German will read and retell stories including novel words on three different days. Idiosyncratic properties of the speakers known to influence breathing and limb movements, i.e. physical fitness and body shape, will be taken into account as determinant factors of speech production in the context of body motion. To further assess the speech-breathing-limb link, speech tasks will be performed in different movement conditions with no motion, free hand motion, and rhythmic motions of the legs or hands.

Based on this corpus, the link between spoken language, breathing and limb motion will be addressed in four working packages analyzing: a) the impact of idiosyncratic physical properties on body motion, respiration and different linguistic levels ranging from phonetics to syntax; b) the impact of body motions on speech planning, prosodic and segmental properties; c) the coordination between speech, breathing and limb motion using sophisticated time series analyses of synchronizations; and d) the role of limb movements for short- and long-term learning of novel information and vocabulary.

The researchers involved in the two teams have multi-disciplinary profiles and complement each other with expertise in language and cognitive sciences, speech production, multi-modal communication, motor control and learning, engineering and signal processing. Different steps are proposed to spread the findings to different scientific and clinical communities and to a broader public. The results of the project will indeed have an impact on fundamental research with a deeper understanding of spoken language in the context of body motions, but will also be useful for applied research in language and speech education and therapy.

Project coordination

Amélie ROCHET-CAPELLAN (Laboratoire Grenoble Image Parole Signal Automatique)

The author of this summary is the project coordinator, who is responsible for the content of this summary. The ANR declines any responsibility as for its contents.

Partner

ZAS Leibniz-Zentrum Allgemeine Sprachwissenschaft
LIDILEM Laboratoire de Linguistique et Didactique des Langues Etrangères et Maternelles
Gipsa-lab Laboratoire Grenoble Image Parole Signal Automatique

Help of the ANR 507,492 euros
Beginning and duration of the scientific project: August 2018 - 36 Months

Useful links

Explorez notre base de projets financés

 

 

ANR makes available its datasets on funded projects, click here to find more.

Sign up for the latest news:
Subscribe to our newsletter