DS06 - Mobilité et systèmes urbains durables

No conventional imagery for secure urban mobility – ICUB

Advanced driver Assitance systems in bad weather conditions

Polarimetric imaging characterizes the reflection of the light. The use of this feature will allow us to go beyond the problems related to only pattern recognition. The light-material interaction and the relationship between polarization, reflection and fog will be a major asset to address the problem of road scene analysis in reduced visibility conditions . In addition, deep neural networks have shown their superiority for obstacle detection over conventional methods.

Prototype for road obstacle detection in bad conditions

-Demonstrate that the analysis of road scenes by current methods is not efficient in bad weather conditions <br />-Propose a sensor technology based on polarimetric vision and associated AI algorithms for : <br />-Road scenes analysis in bad weather conditions <br />-Estimating the distance of detected road obstacles

-Segmentation
-Depth estimation
-Detection/classification
-Multi-sensor data fusion
-Deep neural networks

-Polarimetric image segmentation
-Polarimetric image augmentation by classical and generative adversarial methods (GAN)
-Depth estimation based on a single image (monodepth)
-Detection of road obstacles in degraded weather conditions by multi-sensor fusion (classical and polarimetric)

-Validation of detection algorithms developed on images acquired at CEREMA (Clermont Ferrand) in very reduced visibility conditions (strong fog, strong rain, night)
-Application of monodepth estimation algorithms on complexe use cases (frontal sun, strong reflections, bad weather)
-Consideration of the temporal consistancy (video)
-Experimentation on local tracks
-Real time detection of road obstacles in weather conditions by their semantics and their distance from the driver
-Large scale experimentation in Paris (Stellantis)

-BLIN, Rachel, AINOUZ, Samia, CANU, Stéphane, et Fabrice Mériaudeau. A new multimodal RGB and polarimetric image dataset for road scenes analysis. In : Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. p. 216-217. 2020
-BLIN, Rachel, AINOUZ, Samia, CANU, Stéphane, et Fabrice Mériaudeau. Road scenes analysis in adverse weather conditions by polarization-encoded images and adapted deep learning. In : 2019 IEEE Intelligent Transportation Systems Conference (ITSC). 2019
-Blin, R., Ainouz, S., Canu, S., & Meriaudeau, F. (2019, July). Adapted learning for polarization-based car detection. In Fourteenth International Conference on Quality Control by Artificial Vision (Vol. 11172, p. 1117218). International Society for Optics and Photonics.
-BLANCHON, Marc, SIDIBÉ, Désiré, MOREL, Olivier, et al. P2D: a self-supervised method for depth estimation from polarimetry. arXiv preprint arXiv:2007.07567, ICPR 2020.
-BLANCHON, Marc, MOREL, Olivier, MERIAUDEAU, Fabrice, et al. Polarimetric image augmentation. arXiv preprint arXiv:2005.11044, ICPR, 2020.
-ZHANG, Yifei, MOREL, Olivier, BLANCHON, Marc, et al. Exploration of Deep Learning-based Multimodal Fusion for Semantic Road Scene Segmentation. In : VISIGRAPP (5: VISAPP). 2019. p. 336-343.

As part of research on advanced driving assistance systems (ADAS), the ICUB project aims at designing and developing a new vision based system able to detect road obstacles, even in critical situations like moving obstacles, presence of reflecting objects or puddles on the road, poor weather conditions or faraway obstacles. We propose in this project a consistent system that includes all necessary steps from data acquisition to the targeted detection. Regarding the imaging system, it will involve a stereo-polarimetric head. Obstacle detection will be implemented through multimodal fusion of polarimetric data and disparity map, as provided by stereovision. The main objective of multimodality is to leverage jointly fine scale discrimination of detected objects thanks to polarimetry and accurate distance evaluation of the obstacles (and hence their level of danger) via the disparity map. The use of non-conventional imaging provides an alternative to existing detection techniques by proposing the detection of surface-based properties rather than relying on gray levels or on the geometric properties of obstacles, as conventional scalar methods do. The ICUB project brings together two research laboratories (LITIS, LE2I) and two industrial partners (STEREOLABS and PSA). Each partner is eager to leverage polarimetric information to address road scene analysis.

Project coordinator

Madame Samia Ainouz (Laboratoire d'Informatique du traitement de l'Information et des Systèmes)

The author of this summary is the project coordinator, who is responsible for the content of this summary. The ANR declines any responsibility as for its contents.

Partner

STEREOLABS STEREOLABS
PSA ID PSA ID
ImViA Laboratoire Imagerie et Vision Artificielle
LITIS Laboratoire d'Informatique du traitement de l'Information et des Systèmes

Help of the ANR 483,525 euros
Beginning and duration of the scientific project: December 2017 - 48 Months

Useful links

Sign up for the latest news:
Subscribe to our newsletter