Edge IA - Edge IA

semantiC segmentatiON oF compLexe soUnd scEnes oN edge deviCEs – CONFLUENCE

Submission summary

The CONFLUENCE project aims to develop artificial intelligence (AI) technologies for sound semantic segmentation of acoustic signals that can recognize sound events and separate/isolate the signals of the sound sources forming semantic entities. This can be seen as an audio extension of the image semantic segmentation task that has been receiving much attention in the AI field.
The objective is to design an embedded AI system that can implement these technologies for two telecom application services: immersive communication and home monitoring/assistance devices. Specifically, by analyzing, separating, and augmenting the separated signals with metadata such as location information, it will be possible to transmit only the necessary sounds from a sound scene and spatially recreate that scene in a remote location, or understand what is happening remotely. This could allow advanced communication services, such as privacy-conscious web conferencing systems and home monitoring/assistance devices/systems, that filter out the sounds of daily life and allow only transmit adult speech and important sounds, or new inclusive communication systems that would allow local and remote participants in international conferences to feel in the same location. In this project, we will mainly address the following issues: (1) the development of sound source separation and target sound extraction technologies as a basis for semantic separation, (2) open source implementation of the developed technology and hardware implementation in embedded devices, (3) dissemination of the technology through a proposal to the International Standardization of Mobile Communication Systems (3GPP) and deployment to privacy-preserving home monitoring devices, (4) Contribution to the acceleration of the academic field by disclosing the data acquired in this project and holding international technology evaluation challenges. Through the combination of scientific and industrial expertise from Japan and France, the CONFLUENCE project will develop innovative sound segmentation solutions for embedded devices.

Project coordination

Nicolas Turpault (Sonaide)

The author of this summary is the project coordinator, who is responsible for the content of this summary. The ANR declines any responsibility as for its contents.

Partner

LIST Laboratoire d'Intégration des Systèmes et des Technologies
SON Sonaide
LORIA Laboratoire lorrain de recherche en informatique et ses applications

Help of the ANR 419,707 euros
Beginning and duration of the scientific project: November 2023 - 48 Months

Useful links

Explorez notre base de projets financés

 

 

ANR makes available its datasets on funded projects, click here to find more.

Sign up for the latest news:
Subscribe to our newsletter