Fair Multimodal Learning – FAMOUS

The aim of this project is to explore the first avenues of research into the contribution of multimodality in datasets to meet the requirements of fair learning. Fairness refers here to the biases (in the data and/or induced), while being interested in the interpretability of the models to help their certification.
Each modality has its own statistical and topological characteristics, which requires upstream research on the adjustment of distributions when biased, adapted metrics, etc. Moreover, each one being itself a bias of observation of the data, this will be taken into account to establish a joint distribution (trans-modal) unbiased on all these modalities.
With theoretical research in cross-modal statistical learning, we will study methods for reducing some types of identified biases (non iid, imbalances, sensitive variables) in the case of multimodal data.
Two levels of treatment are privileged: (1) cross-modal pre-processing of biases in the data, by learning metrics, neural representations, and optimization constraints on kernel pre-images; (2) cross-modal algorithms for eliminating biases in model learning: cross-modal optimization algorithms, as well as optimal transfer and transport approaches between modalities to debias the concerned ones, based on the theoretical results previously obtained. Parsimony will be considered for scaling and explainability.
Transversally, our work will be based on problems arising from real data sets in biology and health, multi-modal and presenting various types of bias, and on toy data sets to be generated. They have modalities where the data are structured in graphs: all our fundamental works will be declined to take into account this specificity impacting the treatment of the considered biases.