Bilevel Optimization for Simulations-based Inference – BONSAI

Despite impressive progress in machine learning, it is still challenging to systematically and reliably leverage these methods
in new disciplines to accelerate scientific discovery.
A key challenge is the lack of a consistent way to incorporate domain knowledge produced by experts, such as simulators, into current machine learning frameworks.
Project BONSAI is a multi-disciplinary project aimed at addressing this challenge starting from theory to concrete applications by combining domain knowledge, in the form of simulators, with deep learning through bilevel optimization.

Simulators, such as particle physics simulators or molecular dynamics simulators, are algorithms developed by domain experts to reproduce experimental data. They are often the result of years of work and are typically irreplaceable.
Simulators usually depend on a parameter that must be inferred from data to correctly reproduce and interpret a given experiment.
The task of simulation-based inference (SBI) consists in inferring such a parameter from experimental data by verifying their consistency with simulations.
Performing inference with high precision can have large implications for scientific discovery, e.g. by corroborating or refuting a theory in physics.
However, the inference task is often challenging due to the complexity of simulators for which many quantities of interest, such as the likelihood, are intractable.

We advocate that bilevel optimization methods provide a natural framework for approximating these intractable quantities using deep learning while simultaneously performing the inference task.
Bilevel formulations allow combining several objectives in a flexible fashion that is particularly convenient in the context of SBI.
Project BONSAI aims to leverage the flexibility of bilevel formulation for integrating domain knowledge with deep learning while overcoming the technical challenges of bilevel optimization.