DS06 - Mobilité et systèmes urbains durables

Visual semantic analysis: toward a smarter urban navigation – AVISE

The objectives of this project are to find a solution to the problem of robustness and to the implementation of the extraction of objects of interest for mapping but also for navigation. This project is part of the resolution of a navigation problem of type Localization and simultaneous SLAM mapping. We propose to no longer consider low-level geometric bitters (corners, points... ) but to focus on high-level semantic landmarks directly representing objects such as road signs, cars, etc...

To do this, we use advances in the field of AI, particularly in neural networks and deep learning. The latter has become the state of the art in the field of computer vision, particularly for the recognition of shapes or objects in images. We used these techniques to extract the high-level landmarks present in the data from the vehicle’s sensors and proposed the integration of these complex landmarks into an iterative estimation process.

The results are promising.

We have developed an urban furniture detector (focused on road signs) allowing the precise, reliable and fast extraction of the parameterized contours of their geometric shapes.

Based on this detection, we showed the possibility of obtaining a 3D representation of these objects by triangulation constrained multiple observations.

Finally, we integrated these detections into an iterative estimation process of the Bayesian filtering type and studied robustification methods of these types of approaches on simple modeling cases.

The perspectives of this work are:
- Large-scale and real-data validation of our approach;
- Extension of Bayesian estimation robustification techniques to more realistic models;
- Integration of semantic objects other than panels (e.g. floor markings);

Ces travaux ont conduit aux publications suivantes :

E. Hrustic, R. Ben Abdallah, J. Vilà-Valls, D. Vivet, G. Pagès and E. Chau-mette«Robust Linearly Constrained EKF for Mismatched Nonlinear Systems,International Journal of Robust and Nonlinear Control, 2020.

E. Hrustic, and D. Vivet.«Using Traffic Signs as Landmarks in Object-oriented EKF-SLAM«The 16th International Conference on Control, Automa-tion, Robotics and Vision (ICARCV 2020), December 13 -15, 2020, Shenzhen,China

E. Hrustic, and D. Vivet.«Quadric-based Traffic Sign Landmarks Initializa-tion for Object-oriented EKF-SLAM«2020 ASILOMAR , November 1-5, 2020,Virtual Conference

E. Hrustic, Z. Xu and D. Vivet.«Deep Learning Based Traffic Signs Boun-dary Estimation«2020 IEEE Intelligent Vehicles Symposium (IV’2020), Octo-ber, 2020, Las Vegas, USA

Submission summary

This project joins within the framework of the societal challenge 6 " Mobility and sustainable urban systems " of the generic call for projects of the ANR 2017 in conformance with the instrument of financing Young Researcher ( CJC). It is interested in the improvement of the positioning for the navigation of the urban vehicles.

This project is interested in the implementation of a geolocalized map of objects and semantic events adapted to the urban navigation. The peculiarity of these works lies in the implementation of a bidirectional strong interaction between the mapping and the semantic navigation in an innovative approach of simultaneous localization and semantic mapping (SLAM) using a tight fusion between a semantic visual approach and a GNSS positioning.

Thus the objective is to study the possibilities offered by the active collaboration of a classical GNSS/IMU/Visual Odometry and of a SLAM approach based only on semantic objects for the improvement of the positioning in urban areas in the context of the autonomous navigation. This project so attacks two main aspects of an accurate navigation in urban areas: the precise positioning and the scene analysis and understanding.

The ambition of this project is to integrate different levels of intelligence within the framework of the autonomous navigation, namely to integrate the semantic information into the low level navigation task (only based on the geometrical structure of the scene). For that purpose, surrounding area of Machine-Learning will be integrated to detect, extract and identify static and dynamic objects of the environment more easily recognizable on the long term or according to the weather or visibility conditions.

A higher level of intelligence could also be studied concerning the analysis of the inter-object's interactions to include a certain semantic of action (accidents, pedestrians' crossing etc.).

The long-term ambition is to create an intelligence of the situation, allowing a safe navigation in urban context.

Project coordinator

Monsieur Damien Vivet (Institut Supèrieur de l'Aéronautique et de l'Espace)

The author of this summary is the project coordinator, who is responsible for the content of this summary. The ANR declines any responsibility as for its contents.


ISAE-Supaéro Institut Supèrieur de l'Aéronautique et de l'Espace

Help of the ANR 165,461 euros
Beginning and duration of the scientific project: - 42 Months

Useful links

Sign up for the latest news:
Subscribe to our newsletter