Compressed images and Video interpolation and enhancement for legal evidence – IMPROVED

Several complementary activities are carried within the project:
- Scientific Police user requirements collection
- Relevant databases collection as well as data generation using generative AI, for matching with user needs and cases
- Identification of video codecs and parameters used in real life, in particular images nad data coming from social networks
- Study of legal aspects applicable to artificial intelligence systems, and their effect on fundamental rights
- Performance analysis of image processing based methods for super-resolution,denoising and deblurring
- Benchmark of state of the art AI based super-resolution architectures and models
- Visual attention study

A first milestone was reached with the user needs definition and the collection of databases. All work progressed as expected except a part of WP4, on all of the various study axes of the project. Then an important was to have achieved relevant results for being presented at WISG2024 workshop where all of the project partners were present.
At Month 18 we have legal elements enabling setup of technical tools, aiming at preserving legal evidence, and use of algorithms in respect to regulation, including AI based tools. First versions of the enhancement and restauration tools show very promising results on licence plates, that are one of the targeted objects by users from the Scientific Police

Achieved results show significant progress for super-resolution and deblurring of licence plates. Benchmarks revealed strength and weaknesses of the various existing models and architectures and the subjective study provided a better understanding of the quality perception from the users.

1. Handheld Burst Super-Resolution Meets Multi-Exposure Satellite Imagery. Jamy Lafenetre, Ngoc Long Nguyen, Gabriele Facciolo, Thomas Eboli. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Jun 2023, Vancouver, Canada. ssa.hal.science/hal-04136450/
2. Collaborative Blind Image Deblurring. Thomas Eboli, Jean-Michel Morel, Gabriele Facciolo. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Jun 2024, Seattle, USA. hal.science/hal-04136505
3. Abderrezzaq Sendjasni, Mohamed-Chaker Larabi. «Embedding Similarity Learning for Extreme License Plate Super-Resolution«, IEEE 26th International Workshop on Multimedia Signal Processing (MMSP), Purdue University in West Lafayette, USA

IMPROVED targets studies and tuning of a new generation of tools dedicated to image and video enhancement, for police investigations, while keeping their ability to be used as evidence and respecting individual liberties. The project aims at bridging the gaps encountered by Scientific Police in the field of video enhancement for getting relevant evidence. New video sources, in addition to video surveillance cameras are now available, such as images from smartphones, dashcams or wearable cameras. These devices bring their own sensor characteristics, and video coding features and parameters, thus increasing the difficulties. Enhancement algorithms, especially those relying on Artificial Intelligence, set questions regarding their receivability as legal evidence, due to their black box like design. Image enhancements must consider rights and fundamental principles, in particular loyalty of the legal evidence, and guaranty explainability of the used algorithms. For this, in IMPROVED an interdisciplinary work will be performed between algorithmists and lawyers of the project, with the aim to check, for each enhancement algorithm, whether the processing did not alter the capacity of images to be considered as legal evidence. The project will study new algorithmic directions, such as explainable AI, for making system behaviour more understandable, while providing explanations at each processing step. These algorithmic studies target going beyond the state-of-the-art in image enhancement, while addressing challenges related to new generations of video coding and image sources.