Multimodal perception and reasoning for transnational intelligent vehicles. – PRETIV

PRETIV: Multimodal Perception and REasoning for Transnational Intelligent Vehicles.
HEUDIASYC (CNRS), PKU-KLMP (Peking University), e-Motion (INRIA), PSA Peugeot Citroen. 199990€ + 1000000RMB. 01/2012-12/2014.
Safety and driving assistant systems for a vehicle require advanced technologies in terms of perception and understanding of its external environment. PRETIV is structured with 4 tasks:
- T0: Coordination, management, dissemination + an Advisory Board;
- T1: Multimodal perception;
- T2: Reasoning and scene understanding;
- T3: Test bed integration, experiment design and comparative studies.
Objectives, expected results and innovative nature of PRETIV are summarized as follows:
- Development of an online multimodal perception system which incorporates vehicle localization, mapping of static objects of its environment, detecting and tracking moving object in probabilistic frameworks through multimodal sensing data and knowledge fusion;
- Development of offline reasoning methods based on sensed data, in order to learn semantics, activity and interaction patterns (vehicle – other objects, vehicle – infrastructure), which could be injected as the contextual and country-specific a priori information to make the online perception more general for international use. This analysis aims at comparing perception methods in different traffic environments, and aims at identifying better context-specific a priori information leading to more effective perception algorithms. It includes traffic situation awareness, i.e. generation of indicators to help decision, to estimate the dangerousness of the driving situation by considering factors of sensing uncertainty and infrastructure;
- Cross-experiments and comparative studies are to be conducted in real transnational traffic situations (France and China) by implementing the jointly developed core algorithms on 2 different test-bed vehicles of the partners, which will permit cross-fertilization of sensing techniques;
- A comparative dataset with the experimental data, training sets in typical traffic scenarios in France and China, and ground-truth, which will be used in learning country-specific traffic semantics, and will be opened to public to promote comparative researches on transnational traffic semantics and behaviors. To our knowledge, no such dataset has been proposed for international use.
Added value of the collaboration:
- Innovation and technological transfer on a mutual benefit basis; bi-directional transfer of knowledge and techniques: embedded systems in test-bed vehicles + learning and reasoning techniques.
- Multimodal sensing data from Chinese and French test-bed vehicles will be analyzed, with the objective to go towards more general and better-parameterized perception models for transnational use.
- Open comparative dataset for transnational traffic scene understanding. The dataset will be made available to promote researches towards understanding real-world traffic scenes and cross-country comparative studies.
- Networking: the 3 partners are members of research networks (in Europe, Japan, etc.) that may benefit to each other, providing dynamic linkage with other projects and researchers/students.
The project is based on the solid experience of laboratories in France and China, and is voluntarily oriented towards cross-fertilizations of methods and knowledge. For this purpose, considering their respective experience, the partners will put emphasis on sharing methods such as learning under partial supervision, multi-task learning (that we consider well adapted to the transnational problem with different traffic environments) and data fusion including probabilistic and belief function based frameworks for handling incompleteness and uncertainties in the models and the measures.
An Advisory board will be made up of PSA Peugeot Citroen Research centers in Paris and in Shanghai. It will bring in the point of view, constraints and advices from a car manufacturer.