Autonomous navigation among personal mobility devices – ANNAPOLIS

The overall methodology is based on the paradigm of augmented perception and the hybridization of model-based and data-driven approaches, complemented by decision-making under uncertainty, to solve the problem of safe autonomous driving in the face of unforeseen events and risky behaviors by other road users.

The project duration, initially set at four years, was extended by six months.

The project comprises five work packages:

WP1: Management

WP2: Collaborative Perception

WP3: Detection and tracking of VRUs/PLEVs – Motion prediction

WP4: Decision-making and control

WP5: Integration and evaluation

The main results of the ANNAPOLIS project concern:

Annotation-frugal learning for Roadside Units (RSUs). A self-annotation method for RSU point clouds was developed to reduce the cost of generating training data.

Collaborative fusion and robustness. A weighted late-fusion approach groups bounding boxes from multiple agents using a Breadth-First Search (BFS) based on orientation-penalized 3D IoU, then calculates a consensus box weighted by confidence scores.

Vulnerable road users and heterogeneous data. A hybrid architecture first enriches the ego-vehicle's point cloud with late detections, performs intermediate fusion, and then refines proposals on the collaborative Bird's-Eye View (BEV) map.

Vehicle trajectory prediction at intersections. CG-Net improves trajectory prediction in urban intersections by integrating "candidate centerline" information associated with each vehicle, alongside an interaction encoder inspired by human behavior.

Trajectory prediction for heterogeneous agents. CG-MOVE is capable of generating diverse, accurate, scene-level predictions for agents of various types (e.g., pedestrians, e-scooters). Our approach relies on a hierarchical Mixture-of-Experts (MoE) module to generate probabilities for different scenarios.

Autonomous navigation in the presence of unexpected events. A global probabilistic approach involving multiple controllers is integrated into a reliable system architecture for risk assessment and management. It introduces a decision-making and control strategy based on a multi-level trajectory optimization method, which accounts for uncertainties regarding the movement of surrounding agents through the fusion of predictive stochastic inter-vehicle distance profiles (F-sPIDP).

Context-aware autonomous navigation: By integrating high-definition (HD) maps based on the Lanelet2 format with real-time perception data, the space surrounding a road user is dynamically characterized. This integration leverages the graph structure, semantic richness, and modularity of Lanelet2 maps to provide a comprehensive, context-aware representation of the environment. The goal is to interpret the road from the user's perspective by extracting navigation-relevant information, thereby fostering adaptive and proactive decision-making; this enhances the vehicle's situational awareness and its ability to navigate safely and efficiently in urban scenarios.

Dataset creation: In 2024 and 2025, we created datasets for navigation in the presence of e-scooters and pedestrians.

Beyond the improvements to the contributions made within the ANNAPOLIS project, the following points should be considered.

Datasets: The number and diversity of available datasets are currently insufficient to train data-driven models. It is important to have tools for extracting contexts and scenarios—illustrating specific situations involving Personal Light Electric Vehicles (PLEVs)—and transforming them into configurable virtual scenarios to enrich training datasets.

Multi-modal perception is a key challenge for addressing long-term navigation issues. It is just as important to rethink the content and structure of the environment representation in which the autonomous vehicle operates as it is to study multi-sensor fusion methods capable of effectively handling challenging data acquisition conditions (adverse weather, day/night cycles, etc.) for essential automated or autonomous driving functions.

FM/LLM/VLA: Since the launch of the ANNAPOLIS project, new methods, tools, and approaches have emerged. It would be worthwhile to rethink the situation analysis and decision-making architecture to integrate these new approaches while ensuring the systems remain explainable, ethical, and responsible.

Decoder/Transformer/LLM: New, more efficient networks are now available, making it possible to consider more resource-efficient implementations.

Attention/intention: Attention map extraction and intention prediction are two issues that have not yet been sufficiently addressed.

Social acceptance: Societal acceptance of autonomous vehicles depends on communication efforts—not only between the autonomous vehicle and its passengers but also with other users of the navigable space in which the vehicle operates.

Urban centers are increasingly invaded by new means of individual electric transport (electric scooters, Hoverboard, Gyro-wheel, ...) at the source directly or indirectly of new erratic and unpredictable behavior in the traffic environment. The "Mobility Act 2019" provides for the use of scooters to be used on the lane when the lanes dedicated to the bicycle are not present. In such a context, autonomous vehicles suffer from their perception obtained from onboard sensors and sometimes reduced in the measurement field by bulky obstacles (buses, trucks, ...). ANNAPOLIS aims to increase the perception capacity of the vehicle in the aspects of precision, measurement field and content of the extracted information, to search for new models to take into account these unexpected events, to interpret and analyze scenes in constant evolution, to decide and select the best future movement.