Articulate anything – ARTANY

The Articulator Anything project aims to reconstruct the three-dimensional dynamic evolution of the vocal tract for any language and any speaker. It falls within the field of articulatory synthesis, modeling and simulating the physical process of human speech production using advanced artificial intelligence methods. Current approaches are limited as they rely on static representations of phonemes and fail to capture the temporal dynamics essential for coarticulation and anticipation in natural speech.

The main innovation lies in the multimodal integration of high-resolution real-time MRI data, detailed anatomical acquisitions, synchronized acoustic signals, and precise phonetic annotations. The project is based on three hypotheses: (1) it is possible to reconstruct the full three-dimensional dynamics of the vocal tract for languages absent from existing corpora by combining various data sources; (2) automatic exploitation of these data is feasible thanks to advanced segmentation algorithms; (3) it is possible to normalize speakers' anatomy to merge the dynamics of articulatory gestures despite linguistic and individual variability.

The interdisciplinary methodology combines medical imaging, signal processing, deep learning, experimental phonetics, and automatic speech processing. Potential outcomes are vast: improvement of multilingual speech synthesizers, applications in surgery, speech therapy, linguistics, and enrichment of resources available to the scientific community through the dissemination of developed data and tools.

The consortium brings together experts from the IADI, MultiSpeech, and LPP laboratories, possessing unique expertise in France and Europe to carry out this ambitious project.