The fields of high dimensional statistics, modern inference and machine learning have witnessed transformative changes in the last few years. In particular, recent algorithmic advances on deep networks have convincingly shown that it is possible to learn non-trivial features in data in an unsupervised way. However these exciting advances are almost all empirical. Although the new approaches find their roots in neural network systems (which have a long history) there is very little theoretical basis or understanding for their recent successes at discovering and correctly identifying features or good representations of data.
The aim of the present interdisciplinary project is to benefit from the recent developments in fields as diverse as theoretical statistical physics coding, signal processing, probability and random combinatorial optimization to uncover principles hidden behind the success of the new approaches in modern inference problems.
Our central goals are:
1. Precisely formulate the models of single level and multilevel inference systems in the framework of probabilistic graphical models. Identify the relevant asymptotic limits and construct or generate relevant priors capturing the notion of feature are among our objectives.
2. Thoroughly analyze the phase diagrams of such systems by the methods of statistical physics of spin glasses. Use this analysis to obtain new algorithms and determine fundamental limits to learning features. This includes both information theoretic aspects (the “static” behavior in physics) and algorithmic ones (where the “dynamics” of the algorithm is studied with statistical physics methods)
3. Use the physics predictions to develop a mathematically coherent theory of the algorithmic and optimal phase transitions. Leverage on recent mathematical methods developed in the last few years in the context of coding, signal processing and constraint satisfaction.
Monsieur Florent Krzakala (Laboratoire de physique statistique de l'ENS)
The author of this summary is the project coordinator, who is responsible for the content of this summary. The ANR declines any responsibility as for its contents.
LPS Laboratoire de physique statistique de l'ENS
Communication Theory Laboratory
Help of the ANR 296,676 euros
Beginning and duration of the scientific project: April 2018 - 42 Months