JCJC SIMI 2 - JCJC - SIMI 2 - Science informatique et applications

Interactive Virtual Cinematography – CineCitta

Interactive Virtual Cinematography

L'objectif principal du projet est de proposer et d'évaluer un nouveau processus qui couple des camera trackées avec du calcul automatique de points de vues pour la cinématographie virtuelle. Ce couplage vise à améliorer la créativité des utilisateurs au travers d'une collaboration dynamique entre un réalisateur (ou directeur de la photo) et un outil automatique de planification de points de vues.

Un nouveau processus pour la cinématographie virtuelle intelligente

Les verrous scientifiques et techniques sont: <br />(i) la capacité de générer des points de vues (suggestions) en suivant les règles et conventions de la cinématographie <br />(ii) la capacité à formaliser et représenter des éléments caractéristiques de style et de genre cinématographique <br />(iii) l'intégration de caméra trackées dans le processus

Approche:
(i) fournir un modèle calculatoire efficace et expressif pour formaliser un large ensemble de propriétés charactéristiques de composition;
(ii) proposer une méthodologie pour extraire des éléments caractéristiques de style et de genre cinématographique;
(iii) proposer des métaphores d'interactions pour des scènes complexes avec des caméra trackées;
(iv) rassember des retours de professionnels et évaluer nos approches au travers de collaborations avec des écoles de cinéma

RAS

RAS

Ongoing submissions (SCA, Pacific Graphics)

The main objective of this proposal is to propose and evaluate a new workflow which mixes user interaction using motion-tracked cameras and automated computation aspects for interactive virtual cinematography that will better support user creativity. We propose a novel cinematographic workflow that features a dynamic collaboration of a creative human filmmaker with an automated virtual camera planner. This process aims at enhancing the filmmaker’s creative potential by enabling very rapid exploration of a wide range of viewpoint suggestions of a 3D environment. It looks at enhancing the quality and utility of the automated planner’s suggestions by adapting and reacting to the creative choices made by the filmmaker. This requires three advances in the field. First, the ability to generate relevant viewpoint suggestions following classical cinematic conventions. The formalization of these conventions in a computationally efficient and expressive model is a challenging task in order to select and propose the user with a relevant subset of viewpoints among millions of possibilities. Second, the ability to analyze data from real movies in order to formalize some elements of cinematographic style and genre. The issue here is to encode these characteristic elements of style and genre and let the users select which genre they prefer on a given scene. In this project we propose to characterize elements of style and genre using reinforcement learning techniques from hand-annotated real movies. Third, the integration of motion-tracked cameras in the workflow. Motion-tracked cameras represent a great potential for cinematographic content creation. However given that tracking spaces are of limited size, there is a need to provide novel interaction metaphors to ease the process of content creation with tracked cameras. Finally we propose an evaluation of our tool involving both professionals and film schools.

Proposing tools to interactively assist users in the design of shots, edits and camera paths is a novel and ambitious research path which strongly contrasts with main research in the field, more involved in the design of declarative viewpoint computation, optimization and planning techniques. Furthermore this project displays a strong potential for industrial transfer.

Project coordinator

Monsieur Marc Christie (Université de Rennes 1 / Institut de Recherche en Informatique et Systèmes Aléatoires) – marc.christie@irisa.fr

The author of this summary is the project coordinator, who is responsible for the content of this summary. The ANR declines any responsibility as for its contents.

Partner

Université de Rennes 1 / IRISA Université de Rennes 1 / Institut de Recherche en Informatique et Systèmes Aléatoires

Help of the ANR 208,166 euros
Beginning and duration of the scientific project: September 2012 - 36 Months

Useful links