Coupled Learning and Vision for Aerial Robot Control – CLARA

Different approaches and methods are being studied in the work in progress. Among these are the following:
1) the study of optical flow estimation in spherical images based on deep learning methods,
2) The study of taking into account the geometry of the scene using a depth sensor to improve the convolution filters,
3) The study of advanced control strategies for the navigation of aerial vehicles in complex and unstructured environments.

The project is progressing well and the preliminary results obtained are very good. Some of them have already been published in conferences and others are in the process of being written.
Interactions with all consortium members are very good.

The two PhD funded by ANR began in late 2019 and early 2020. The initial objectives are still valid, namely :
- The control of the drone in a complex and unknown environment;
- The perception and representation of a complex and unstructured environment for obstacle avoidance and the study of traversability;
- 3D reconstruction and localization of a flying robot in an environment where objects are not very discriminate / differentiable.

Z. Wu, G. Allibert, C. Stolz, C. Demonceaux, Depth-Adapted CNN for RGB-D Cameras, Asian Conference on Computer Vision (ACCV20), Kyoto, Japan, December 2020
hal.archives-ouvertes.fr/hal-02946902

C.O. Artizzu, H. Zhang, G. Allibert, C. Demonceaux, OmniFlowNet: A Perspective Neural Network Adaptation for Optical Flow Estimation in Omnidirectional Images, in IEEE International Conference on Pattern Recognition (ICPR20), Milan, Italia, January 2021
hal.archives-ouvertes.fr/hal-02968191

I.S. Mohamed, G. Allibert, P. Martinet, Model Predictive Path Integral Control Framework for Partially Observable Navigation : A Quadrotor Case Study, IEEE International Conference on Control, Automation, Robotics and Vision (ICARCV20), Shenzhen of China, December 2020
hal.archives-ouvertes.fr/hal-02545951

A.R. Sekkat, Y. Dupuis, P. Vasseur, P. Honeine, Génération d'images omnidirectionnelles à partir d'un environnement virtuel, GRETSI, 2019.
hal-normandie-univ.archives-ouvertes.fr/hal-02183033

Unmanned aerial vehicles (UAVs), commonly known as drones, are robots now highly used for many applications like monitoring, prevention or site surveillance. In the case of an obstacle-free environment, autonomous navigation can be considered by controlling a vehicle along GPS way-points. For more complex situations such as monitoring engineering structures, a pilot is often necessary to ensure safety during the flight. In even more complex environments such as in a forest, both autonomous and human piloting become impossible due to many constraints :
- The loss of GPS signals is classic in this kind of difficult environment making way-point very complicated if not impossible,
- Dense and unstructured environments (branches, foliage, ...) reduce visibility.
Nevertheless, the use of aerial drones in this type of environment is of great interest and can be broken down into multiple prevention and / or surveillance applications such as fire detection, search for missing people, mapping for forest maintenance, etc.
In this project, we propose to tackle the challenge of autonomous navigation of an UAV within a forest with the added objective of 3D mapping. We then distinguish three scientific hurdles to solve:
- The control of the drone in a complex and unknown environment;
- The perception and representation of a complex and unstructured environment for obstacle avoidance and the study of traversability;
- 3D reconstruction and localization of a flying robot in an environment where objects are not very discriminate / differentiable.
In the CLARA project, to respond to the identified scientific constraints and obstacles, we will assume that the drone does not have a map of the environment or does not receive. Consequently, it has to rebuild a simple map of the environment while moving autonomously towards a predefined position. For this, the drone will be equipped with an inertial unit and a stereoscopic vision system consisting of two 360-degree cameras.