No conventional imagery for secure urban mobility – ICUB

-Segmentation
-Depth estimation
-Detection/classification
-Multi-sensor data fusion
-Deep neural networks

-Polarimetric image segmentation
-Polarimetric image augmentation by classical and generative adversarial methods (GAN)
-Depth estimation based on a single image (monodepth)
-Detection of road obstacles in degraded weather conditions by multi-sensor fusion (classical and polarimetric)

-Validation of detection algorithms developed on images acquired at CEREMA (Clermont Ferrand) in very reduced visibility conditions (strong fog, strong rain, night)
-Application of monodepth estimation algorithms on complexe use cases (frontal sun, strong reflections, bad weather)
-Consideration of the temporal consistancy (video)
-Experimentation on local tracks
-Real time detection of road obstacles in weather conditions by their semantics and their distance from the driver
-Large scale experimentation in Paris (Stellantis)

-BLIN, Rachel, AINOUZ, Samia, CANU, Stéphane, et Fabrice Mériaudeau. A new multimodal RGB and polarimetric image dataset for road scenes analysis. In : Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. p. 216-217. 2020
-BLIN, Rachel, AINOUZ, Samia, CANU, Stéphane, et Fabrice Mériaudeau. Road scenes analysis in adverse weather conditions by polarization-encoded images and adapted deep learning. In : 2019 IEEE Intelligent Transportation Systems Conference (ITSC). 2019
-Blin, R., Ainouz, S., Canu, S., & Meriaudeau, F. (2019, July). Adapted learning for polarization-based car detection. In Fourteenth International Conference on Quality Control by Artificial Vision (Vol. 11172, p. 1117218). International Society for Optics and Photonics.
-BLANCHON, Marc, SIDIBÉ, Désiré, MOREL, Olivier, et al. P2D: a self-supervised method for depth estimation from polarimetry. arXiv preprint arXiv:2007.07567, ICPR 2020.
-BLANCHON, Marc, MOREL, Olivier, MERIAUDEAU, Fabrice, et al. Polarimetric image augmentation. arXiv preprint arXiv:2005.11044, ICPR, 2020.
-ZHANG, Yifei, MOREL, Olivier, BLANCHON, Marc, et al. Exploration of Deep Learning-based Multimodal Fusion for Semantic Road Scene Segmentation. In : VISIGRAPP (5: VISAPP). 2019. p. 336-343.

As part of research on advanced driving assistance systems (ADAS), the ICUB project aims at designing and developing a new vision based system able to detect road obstacles, even in critical situations like moving obstacles, presence of reflecting objects or puddles on the road, poor weather conditions or faraway obstacles. We propose in this project a consistent system that includes all necessary steps from data acquisition to the targeted detection. Regarding the imaging system, it will involve a stereo-polarimetric head. Obstacle detection will be implemented through multimodal fusion of polarimetric data and disparity map, as provided by stereovision. The main objective of multimodality is to leverage jointly fine scale discrimination of detected objects thanks to polarimetry and accurate distance evaluation of the obstacles (and hence their level of danger) via the disparity map. The use of non-conventional imaging provides an alternative to existing detection techniques by proposing the detection of surface-based properties rather than relying on gray levels or on the geometric properties of obstacles, as conventional scalar methods do. The ICUB project brings together two research laboratories (LITIS, LE2I) and two industrial partners (STEREOLABS and PSA). Each partner is eager to leverage polarimetric information to address road scene analysis.