PICNICS: PoInt Cloud compressioN for human perceptIon and maChine viSion – PICNICS

The main objective of the PICNICS project is to build an efficient end-to-end compression scheme dedicated to static and dynamic 3D point clouds, for both machine and human vision tasks. It will rely on the most recent artificial intelligence techniques to propose powerful solutions to the challenges related to 3D-PCs. PICNICS will design a proper encoding function for 3D-PCs with the aim of producing appropriate and powerful features for both tasks. This includes a smart resource allocation strategy in the latent space between the base layer features (analysis) and enhancement features (reconstruction). To tackle the dynamic aspect of 3D-PCs, a novel solution focusing on temporal feature extraction will be proposed. A major target is to make the used learning models weakly/self-supervised and label-frugal. Another objective is to define a holistic quality evaluation solution for end-to-end quality assurance of 3D-PCs. This includes the definition of a dedicated subjective assessment paradigm for extended-reality scenarios in addition to perceptually based quality metrics dedicated to static and dynamic PCs. At the application side of PICNICS, different machine vision use-cases will be investigated on static and dynamic PCs including object/place recognition, video surveillance as well as autonomous driving. In these use-cases, leveraging different a priori knowledge including multi-modality and spatiotemporal redundancy allows designing self-supervised learning models that suit both compression and general vision utility tasks. Finally, making the proposed models lightweight and real-time is also a major challenge, particularly for dynamic PCs. All the use-cases designed in PICNICS will be evaluated using available benchmark datasets.