Statistical Guarantees for Uncertainty Set Selection in Distributionally Robust Sequential Decision-Making – USS-RSD

This project aims to develop statistically certified methods for selecting uncertainty sets in robust Markov decision processes (RMDPs), which are used for sequential decision-making in uncertain environments. RMDPs consider that unknown parameters, such as transition probabilities, are adversarially chosen from an uncertainty set. The concept of "rectangularity" of the uncertainty set is central, representing the independence of uncertain parameters across different states and actions in the system. Despite extensive research on the computational aspects of RMDPs for specific types of uncertainty sets, the question of how to select uncertainty sets with both computational tractability and statistical efficiency, given limited data, has been less explored.

The project has three main objectives:

1. To determine which uncertainty model (sa-rectangularity, s-rectangularity, non-rectangularity) offers the best statistical guarantees when only limited historical data is available. The goal is to compare the models in terms of consistency, efficiency, and generalization, in order to identify the model that provides robust optimal policies with the least amount of data.
2. To develop statistical methods that can detect the appropriate level of rectangularity directly from the observed data. This allows transitions to be modeled more realistically by capturing latent dependencies.
3. To explore the possibility of defining a hierarchy of models that interpolates between different levels of rectangularity, making decision policies more flexible and better suited to the structure of the data.

These topics will be studied in the case of a single source of data (Objective 1) and in the case of a dataset aggregating multiple data sources (Objective 2), with a particular focus on preserving confidentiality.
The project is a collaboration between Julien Grand-Clément (HEC Paris) and Nian Si (HKUST), supported by Ph.D. students (HKUST) and a postdoctoral researcher (HEC), to be hired. Together, they will develop theoretical and empirical methods to guide robust decision-making in real-world contexts where data is uncertain and limited.