DS09 - Liberté et sécurité de l’Europe, de ses citoyens et de ses résidents

Computer Vision for Automated Holistic Analysis of Humans – ENVISION

Submission summary

This proposal is motivated by the rapid evolution in the volume, complexity and utility of biometrics. While in the past, biometrics was mainly used for person identification by employing simple matching of well-defined single biometric traits (such as face, iris or fingerprint), we are now – rather heuristically – transitioning to large-scale biometric systems that involve many traits and many tasks that often include deducing ancillary information beyond identity, or even estimating mental and psychological states.
This proposal focuses mainly on elevating biometrics to a stochastic setting that reflects the dynamic and error-prone nature of modern automated-security systems, thus aiming to align biometrics with powerful tools from machine learning and big data. Our work will seek to derive the fundamentals of evolved biometric systems that must now operate – not in the idealistic and sanitary settings previously associated to classical biometrics – but rather in the presence of severe operational randomness associated to noisy sensors, as well as in the presence of substantial data uncertainty.
Thus the grand challenge of ENVISION is to lay algorithmic and theoretic foundations of large-scale biometric systems, which describe humans in a holistic manner and do so in the presence of operational randomness and data uncertainty. To achieve this, our proposal advocates for a macroscopic research path, which consolidates the most foundational aspects of complex biometric systems, taking into consideration their new salient features of data massiveness and uncertainty, multi-trait fusing, operational uncertainty, and multitude of uses.
Part of our contribution will be on designing computer vision algorithms analyzing appearance and dynamics of face and body, towards recognition of identity, gender, age, as well as mental and social states of humans in unconstrained settings. Such dynamics – which include facial expressions, visual focus of attention, hand and body movement – constitute a new class of tools that will allow for (a) successful biometric identification in the presence of difficult operational settings that cause traditional traits to fail, (b) efficient analysis of elderly subjects for automated healthcare.
In terms of algorithms, ENVISION aims to design, deploy and assess low-cost unobtrusive sensing environment for mental and social human states in unconstrained settings. In terms of theory, ENVISION also aims to lay theoretical foundations for large-scale biometric systems, exploring fundamental limits of algorithms, and employing computer vision, simulations and mathematics to reveal the true information content and uniqueness/distinctiveness/similarity of different traits and of different combinations of traits. All our efforts will be validated by extensive deployment of these algorithms in hospital and supermarket experimental settings currently working with the PI’s team.
What we propose has never been done: Despite the obvious urgency, the proposed underlying foundations do not exist, and the corresponding solutions are often heuristic and can lack validation and insight. Our approach has the potential to offer maturity to a new field of large societal impact, and to thus allow us to rigorously measure large-scale biometrics as a function of impact, security, cost, and privacy. It is only then that large-scale biometric systems will become an effective tool in the effort to balance safety and manage risk, while preserving privacy and individual freedom.

Project coordination

Antitza DANTCHEVA (Centre de Recherche Inria Sophia Antipolis - Méditerranée)

The author of this summary is the project coordinator, who is responsible for the content of this summary. The ANR declines any responsibility as for its contents.


INRIA Sophia Antipolis Centre de Recherche Inria Sophia Antipolis - Méditerranée

Help of the ANR 217,126 euros
Beginning and duration of the scientific project: - 36 Months

Useful links

Explorez notre base de projets financés



ANR makes available its datasets on funded projects, click here to find more.

Sign up for the latest news:
Subscribe to our newsletter