Simulation Analytics and Metamodel-based solutions for Optimization, Uncertainty and Reliability AnalysIs – SAMOURAI

The first scientific challenge of the project focuses on the design of metamodels adapted to large scale problems (typically around 100 input variables) in the context of a limited budget of simulations (around 500).

The second objective is to adapt sequential enrichment strategies to large scale problems for reliability-based design optimization and inversion purposes. This calls for defining simplified acquisition functions and designing adapted methods for their optimization.

Some real applications of the SAMOURAI project are optimal design problems mixing variables of different types: continuous, ordinal and nominal. The third objective of the project is thus to design efficient black-box optimization methods capable of handling this type of variables.

The fourth objective is to increase the performance of the iterative process (optimization and MM building) in case of instabilities, failures or non-physical results of the simulation workflow: the objective will be to learn the hidden constraints and deal with them in the adaptive design procedure.

In this first phase of the project, several test cases were set up: 2 cases of turbine blade design (Safran), one case of wind turbine placement in a farm (EDF), and one case of wind turbine reliability analysis (IFPEN).

We describe the main results obtained for the 4 challenges
Challenge 1: High-dimensional meta-models
• A Bayesian extension of the non-parametric regression model COSSO (Component Selection and Smoothing Operator) has been proposed and is based on a hierarchical Gaussian process model (GP). The parameters of this model allow to define a Bayesian feature selection model.
• HSIC-ANOVA indices to select and prioritize the most influential input variables: a theoretical study has demonstrated some properties of the (Sobolev) kernels associated with HSIC-ANOVA indices: characteristic kernel, and identification of several feature maps associated with kernels. Based on these properties, independence tests for variable selection have been developed, with a power in practice at least as great as that of the usual HSIC tests.
• Improvement of the estimation of hyperparameters by a constrained multi-objective optimization algorithm, taking into account not only the likelihood of the data, but also criteria related to the quality of the predictive distribution of the metamodel.
Challenge 2: Efficient enrichment strategy for inversion under uncertainty
• A literature review on optimization and inversion for reliability problems was conducted.
• A sampling criterion has been developed for inversion problems for reliability and is being evaluated on a turbine blade design application.
Challenge 3: Meta-models and optimization with continuous and categorical mixed variables
• A study of kernels for mixed-input PGs has been carried out by considering the case of point clouds with variable size, in connection with the design of wind farms.
• First applications of these new meta-models based on the proposed kernels have been realized: an application to the computation of the annual production of a wind farm as a function of the location of the turbines with two versions, an analytical toy case and the test case of EDF.
Challenge 4: learning and taking into account hidden constraints
• Archissur, a learning method based on a stepwise uncertainty reduction strategy with PG classification (PGC) models has been developed. The method has been applied to a test case of wind turbine reliability analysis (IFPEN test case).
• The other part of this challenge is devoted to black-box optimization in the presence of hidden constraints, in particular thanks to the MADS algorithm and its implementation NOMAD. Another family of classification models (k-nearest neighbours) was first used, which will be replaced by ARCHISSUR in a second time.

For Challenge 1:
- It is planned to compare the use of HSIC-ANOVA with other feature selection methods (HSIC LASSO, e.g.) to extend the use of PGs in high dimension
- The proposed hyperparameter optimization algorithm will be compared with other recent «robust« estimation methods (use of scoring rules, reparameterization of the likelihood function, Bayesian approaches).

For challenge 2:
- The evaluation of the proposed enrichment criterion continues on applications to other test cases
- Work for an efficient implementation of the sampling criterion, based on SMC (Sequential Monte-Carlo) methods. Depending on the needs of the applications, a planning by batch of points will be studied.

For challenge 3:
- Extend the definition of point clouds by introducing differentiated weights, representing the influence of each wind turbine on the production of all others
- Generalize the choice of kernels on point clouds
- Optimization based on these new metamodels

For challenge 4:
The next phase will consist in coupling the Archissur method for learning hidden constraints to the generation of an optimal experimental design dedicated to the construction of a predictive meta-model and to optimization.

Gabriel Sarazin, Amandine Marrel, Sébastien da Veiga, Vincent Chabridon. Test d'indépendance basé sur les indices HSIC-ANOVA d'ordre total. 53èmes Journées de Statistique de la SFdS, Société Française de Statistique (SFdS); Université Claude Bernard Lyon 1, Jun 2022, Lyon, France. ?cea-03701170?

Babacar Sow, Rodolphe Le Riche, Julien Pelamatti, Sanaa Zannane, Merlin Keller. Gaussian Processes Indexed by Clouds of Points: a study Babacar SOW (EMSE, LIMOS), Rodolphe LE RICHE (CNRS, LIMOS). MASCOT-NUM, Jun 2022, Clermont Ferrand, France. ?emse-03720276>

Romain Ait Abdelmalek-Lomenech, Julien Bect, Emmanuel Vazquez. Sequential Bayesian inversion of black-box functions in presence of uncertainties. MASCOT-NUM 2022, Jun 2022, Clermont-Ferrand, France. ?hal-03694867?

Morgane Menz, Miguel Munoz Zuniga, Delphine Sinoquet. Learning hidden constraints using a Stepwise Uncertainty Reduction strategy with Gaussian Process Classifiers. Optimization days 2022, May 2022, Montréal, Canada. ?hal-03688224?

Amandine Marrel, Bertrand Iooss, Vincent Chabridon. The ICSCREAM methodology: identification of penalizing configurations in computer experiments using screening and metamodel applications in thermal-hydraulics. UQ22 - SIAM Conference on Uncertainty Quantification, SIAM, Apr 2022, Atlanta, United States. ?cea-03700747?

Romain Ait Abdelmalek-Lomenech, Julien Bect, Emmanuel Vazquez. Reliability-based inversion: Stepwise uncertainty reduction strategies?. SIAM Conference on Uncertainty Quantification (UQ22), Apr 2022, Atlanta, United States. ?hal-03694921?

Gabriel Sarazin, Amandine Marrel, Sébastien da Veiga, Vincent Chabridon. What is hidden behind the Sobolev kernels involved in the HSIC-ANOVA decomposition ?. 2022 SAMO Conference - 10th International Conference on Sensitivity Analysis of Model Output, Florida State University, Mar 2022, Tallahassee, United States. ?cea-03701074?

A. Marrel, Bertrand Iooss, V Chabridon. The ICSCREAM methodology: Identification of penalizing configurations in computer experiments using screening and metamodel -- Applications in thermal-hydraulics. Nuclear Science and Engineering, Academic Press, 2022, 196, pp.301-321. ?10.1080/00295639.2021.1980362?. ?hal-02535146v4?

R package Sensitivity, Version 1.28.0, Global Sensitivity Analysis of Model Outputs, CRAN.R-project.org/package=sensitivity

Increasing the efficiency of model-based industrial processes requires to improve their uncertainty quantification and numerical optimization. Such issues appear in most of the engineering domains (e.g. energy, transport, agriculture) and scientific fields (e.g. biology, high-energy physics). A major problem comes from the black-box nature of the process/function of interest that is often not directly accessible: in general, the only available information are the outputs of the black-box simulation workflow. In particular, derivative information, which is very valuable in the context of optimization and uncertainty quantification, does not exist or is not available. This is a direct consequence of the increasing complexity and diversity of the industrial problems to be addressed (e.g. coupling of multi-physics or multidisciplinary simulators, creation of economic models, working with more sophisticated machine learning models, handling of uncertainty and non-Euclidean variables). Solving this problem is therefore a major issue with direct and significant industrial benefits.

In the two last decades, the field of black-box optimization (BBO) methods – especially, derivative-free optimization and surrogate or metamodel (MM) management frameworks – has experienced major theoretical and practical developments. Nevertheless, despite the growing popularity of these methods, some fundamental limitations remain: in particular, the scale of the problems that can currently be efficiently solved by BBO methods does not exceed a few tens of variables and methods to deal with high-dimensional or categorical variables are limited. In real-world applications, the simulation budget is often very restricted. Moreover, BBO algorithms have become complex tools in themselves, which raises questions about their generality of use (choice of kernels for MM) and the reliability of hyper-parameter learning.

The main objectives of the project are, jointly, to develop innovative simulation and surrogate-based optimization methodologies, while pushing back their current limits of performance and applicability, guided by real-world applications. These applications are related to the design and risk assessment of critical and complex systems. Hence, the partners of the project will provide challenging and critical applications in the fields of renewable and low-carbon energies and reduced CO2 air transport domains, in order to demonstrate the relevance and the efficiency of the developed methodologies.
More precisely, the partners’ ambition is to solve the four following major challenges:
- design MM adapted to large scale problems (typically around 100 input variables) in the context of a limited budget of simulations (around 500);
- adapt sequential enrichment strategies to large scale problems for reliability-based design optimization and reliability-based inversion purposes;
- design efficient black-box optimization methods capable of handling problems mixing input variables of different types: continuous, ordinal and nominal variables;
- increase the performance of the iterative process (optimization and MM building) in case of instabilities, failures or non-physical results of the simulation workflow: the aim will be to learn the hidden constraints and deal with them in the adaptive design procedure.
Thus, the project aims at (i) consolidating and extending the existing surrogate-based optimization methods to provide a real improvement of their application to industrial problems, (ii) sharing experiences and methodologies of industrial partners for practical problems, (iii) integrating the resulting methods and methodologies in open-source platforms developed by the partners.