Scalable Nonlinear Data Assimilation with Ensembles – SuNDAE

Our aim is to set up advanced assimilation techniques for improving the prediction of high impact weather, such as fog and thunderstorms. The ultimate goal is that the assimilation scheme is able to infer the balanced three dimensional, thermodynamic structure of the atmosphere from the available observations. We intend to blend the best features of variational methods with the ones of ensemble methods, combining the advantages of these two approaches and avoiding their drawbacks.
The benchmark algorithm for data assimilation in numerical weather prediction is the four-dimensional variational scheme (4DVar). The 4DVar minimizes a cost function over a time window. Non-linear effects are taken into account using advanced formulations of 4DVar. However, the 4DVar algorithm requires the development of tangent linear and adjoint models which is particularly difficult at the mesoscale. The sequential nature of 4DVar raises concerns about the scalability on future computer architectures.
Alternatively, (mostly linear) ensemble methods have become popular in the community, in various algorithmic forms, including the popular Ensemble Kalman Filter (EnKF). Such schemes allow for time and flow-dependent evolution of the analysis and forecast error covariances contrary to standard variational methods. However there are drawbacks to the current algorithmic implementation of EnKF variants, including a lack of scalability and suboptimal use of non-local observations.
Despite recent progress, the question of a four-dimensional, non-linear, ensemble-based assimilation algorithm for applications in high dimensions remains open. The first objective of the SuNDAE project is to evaluate the scientific and computational feasibility of developing such an algorithm. We further plan to evaluate this algorithm for numerical weather prediction with the AROME model.
AROME is the flagship, high resolution numerical model of Météo-France. AROME gathers contribution from the European and in particular the French research community. The development of the AROME towards more advanced data assimilation is possible through the use of the Object-Oriented Prediction System (OOPS), a flexible framework for data assimilation that separates the algorithm from its model-dependent implementation and that is shared by various institutes and universities.
The SuNDAE project will develop a prototype of a non-linear assimilation scheme for the AROME model under OOPS. We plan it to be fully functional in order to allow for extensive comparison at the same spatial resolution. A progress of the AROME performance through a better assimilation scheme will go with the future investment on the Météo-France supercomputer and may be regarded as highly beneficial to the society. This development would valorize AROME, a tool that gather two European Numerical Weather Prediction consortium as well as the french research community. The choice of OOPS will allow the application of the algorithm to other communities such as the ocean and air quality, with reinforced collaborations between institutes.
It is noteworthy that within our ocean-atmosphere community, similar question arise from the continuous increase in scale both in the observation and in the modelization areas. SuNDAE achievements will also benefit this wider community. Contacts are strong between scientists, being either focused on atmosphere, land or ocean, and we will took the various opportunities we have to interact and share the results of this work.