Multimodal tracking and identification of low signature flying targets using Deep-Learning – DEEPLOMATICS

The illegal or malicious use of unmanned aircrafts vehicles (UAVs) represents an emerging threat, which is only partially addressed by existing anti-intrusion systems. The DEEPLOMATICS project proposes an innovative and adaptive solution for the real-time identification and robust tracking of UAVs. The main objective is to address the problem of intrusion detection on critical sites and infrastructures, both in open spaces and in urban areas.

The DEEPLOMATICS consortium proposes a new approach, entirely based on innovative Deep Learning techniques. This approach will allow a robust real-time identification and tracking of low-signature UAVs, thanks to the use of a multimodal and heterogeneous sensors connected to specialized artificial intelligences (AI).

This interdisciplinary project jointly uses advanced localization techniques using scalable networks of microphone arrays, and an optronic active imaging system, each feeding specialized deep neural networks. Each independent smart acoustic devices consists in a compact digital MEMS microphone array, which achieve high directivity in a broadband frequency range. These smart microphone arrays will be complemented by an optronics active imaging system, which will also be connected to an independent AI. This modular and scalable approach will allow us to adapt the sensors topology to the protected sites. We aim at offering an autonomous system, by taking advantage of the convergence of acoustic signal processing, data sciences, and optronics.

The proposed approach will simultaneously enable dynamic localization and automatic recognition of drones, while adapting to the measurement environment and to the topology of the heterogeneous on-site sensor network. The audio and video data will feed deep neural networks connected to each modules. Each AI will be pre-trained for UAV dynamic tracking and identification.

Acoustic and optronic sensors do not operate in the same wavelength range, and they operate autonomously. Deep Learning acoustic strategies for target tracking will allow to identify the target and assess its position with a precise angular localization, over a wide coverage. The overall coverage area offered by this scalable network therefore only depends on the number of smart microphone arrays in the acoustic surveillance network. One of the proposed network topologies allows to cover a 1.7 km diameter surveillance zone. The optronics active imaging system has a narrow viewing angle and has a maximum operational range of 1.5 km. The computer vision methods associated to the active imaging system will allow estimating the targeted UAV distance, while tracking its trajectory in real time, after having locked onto the target thanks to the localization data transmitted by the acoustic network. Target recognition will also benefit from this multimodal and modular approach, thanks to the high contrast of the video feed obtained by active imaging, and the spatial filtering achieved by smart acoustic arrays. One advantage of this solution comes from the complementarity of the two localization modalities. Data fusion of video and audio artificial intelligences will exploit this complementarity in order to address most of intruding scenarios.

Classical acoustic localization approaches, based on propagation models, can be replaced by Machine Learning methods. Our preliminary investigations reveal the superiority of the latter approach in a complex environment, even when the microphone array calibration is difficult. Deep learning approaches implicitly incorporate a robust self-calibration that automatically adapts the AI to the array specificities, even during the life of the on-site sensor. They also allow an automatic adaptation of the localization and identification algorithms to the protected site, and give access to array directivities that are not attainable with classical techniques, while being strongly robust to environmental noise.