CE23 - Intelligence artificielle et science des données

Wasserstein Gradient Flows for Optimization and Sampling: non asymptotic properties and the non log-concave setting – WOS

Submission summary

An important problem in machine learning and computational statistics is to sample from an intractable target distribution. In Bayesian inference for instance, the latter corresponds to the posterior distribution of the parameters, which is known only up to an intractable normalisation constant, and is needed for predictive inference. In deep learning, optimizing the parameters of a big neural network can be seen as the search for an optimal distribution over the parameters of the network.
This sampling problem can be cast as the optimization of a dissimilarity (the loss) functional, over the space of probability measures. As in optimization, a natural idea is to start from an initial distribution and apply a descent scheme for this problem. In particular, one can leverage the geometry of Optimal transport and consider Wasserstein gradient flows, that find continuous path of probability distributions decreasing the loss functional. Different algorithms to approximate the target distribution result from the choice of a loss functional, a time and space discretization; and results in practice to the simulation of interacting particle systems. This optimization point of view has recently led to new algorithms for sampling, but has also shed light on the analysis of existing schemes in Bayesian inference or neural networks optimization.
However, many theoretical and practical aspects of these approaches remain unclear. First, their non asymptotic properties quantifying the quality of the approximate distribution at a finite time and for a finite number of particles. Second, their convergence in the case where the target is not log-concave (which is analog to the non-convex optimization setting). Motivated by the machine learning applications mentioned above, the goal of this project is to investigate these questions, by leveraging recent techniques from the optimization, optimal transport, and partial differential equations literature.

Project coordination

Anna KORBA (Groupe des écoles nationales d'économie et statistique)

The author of this summary is the project coordinator, who is responsible for the content of this summary. The ANR declines any responsibility as for its contents.

Partner

CREST Groupe des écoles nationales d'économie et statistique

Help of the ANR 202,270 euros
Beginning and duration of the scientific project: March 2023 - 36 Months

Useful links

Explorez notre base de projets financés

 

 

ANR makes available its datasets on funded projects, click here to find more.

Sign up for the latest news:
Subscribe to our newsletter