Artificial Vision & Intelligence for Visual Effects – ALICIA-Vision

The photorealistic combination of computer-generated images and film footage images of a same scene is a central objective in the creation of visual effects for the cultural and creative industries. The constant advances in numeric algorithms and computer computing power have made artificial vision (also known as computer vision) a mature discipline so that a large number of dedicated applications are now being used in these industries. While these new technologies have profoundly changed workflows, many tasks remain manual. Very recently, thanks to the increasing prevalence of very large datasets and the ability of computers to process them, advances in artificial intelligence (AI) for machine vision, and in particular machine learning, have created a genuine scientific revolution. These advances make it possible to envisage a new qualitative leap in the performance of tasks required by visual effects, by “learning” computers how to perform these tasks using examples, in order to complete or replace certain "classic” artificial vision algorithms, which today limit the implementation of fully automated processing.
The objective of the ALICIA-Vision joint laboratory is to formalize a joint research activity between the REVA team at IRIT (Institut de Recherche en Informatique de Toulouse) and the post-production company Mikros Image, located in Paris. This activity aims:
to innovate in the field of artificial vision, using geometric and photometric approaches, to extract precise knowledge about cameras, the scene and materials of which it is made, and light sources;
to create a transversal link between the theoretical and algorithmic approaches for the creating visual effects by “classical” artificial vision, as currently developed by REVA and Mikros Image, and approaches to these problems by "intelligent” artificial vision, i.e. according to the paradigms of machine learning in AI, in particular those of deep learning.
The software integration of the research conducted by ALICIA-Vision is intended to expand the libraries of the AliceVision 3D reconstruction pipeline, used in production since 2016 by Mikros Image. This “classic” artificial vision pipeline, distributed as open source, was developed as part of two European Horizon 2020 projects involving the REVA team (formerly VORTEX) and Mikros Image. The main application, called Meshroom, allows graphic designers to interactively configure pipeline processing.
LabCom ALICIA-Vision aims at capitalizing on the AliceVision pipeline and its software environment with ambitious objectives to develop new AI algorithms along four research axes:
axis 1: improving the performance and robustness of the 3D reconstruction pipeline;
axis 2: camera tracking by localization and tracking of moving objects;
axis 3: extraction of geometric and photometric characteristics in a multi-view pipeline;
axis 4: determination of materials in a multi-view pipeline.