Security of AI for Defense Applications – SAIDA

The Villani report emphases the utmost importance of mastering AI in defense and security applications. France must keep an advantage in AI over its adversaries and be technologically on par with its Allies. A dependence to non-European technology providers is a national threat.

The main utility of AI in defense applications is the processing of information. Like in other context, data is plentiful, large, heterogeneous, but here they are also confidential, sensitive, hyperviolent and originating from sources varying in trust. The existence of very serious adversaries fundamentally differentiates defense and security applications. Adversaries can attack systems and manipulate data in order to cause e.g. false negatives (missing an event) or false positives (raising a wrong alarm), overall reducing the performance of an AI-based decision process.

SAIDA targets the AID “Fiabilité de l’intelligence artificielle, vulnérabilités et contre-mesures” chair. It aims at establishing the fundamental principles for designing reliable and secure AI systems: a reliable AI maintains its good performance even under uncertainties; a secure AI resists attacks in hostile environments. Reliability and security are challenged at training and at test time. SAIDA therefore studies core issues in relation with poisoning training data, stealing the parameters of the model or inferring sensitive training from information leaks. Additionally, SAIDA targets uncovering the fundamentals of attacks and defenses engaging AI at test time. Three converging research directions make SAIDA.

First, SAIDA proposes to achieve theoretical investigations grounded in statistics and applied mathematics in order to discover the underpinnings of reliability and security. The resulting understanding, fundamental to most machine learning tasks, will lead to inventing sound training procedures. Second, SAIDA will make the connection between adversarial sampling and Information Forensics and Security which includes image forensics, digital watermarking, and steganography. Applying lessons from IFS is a potential major breakthrough in Machine Learning research and may lead to designing much stronger counter-measures for protecting national technologies as well as new models of attacks able to successfully challenge alien and hostile IAs. Third, SAIDA will study the protection of the training data and the AI system. It is important to prevent any information leakage about sensitive data and to block malicious attempts trying to poison the training set.

SAIDA thus combines theoretical investigations with more applied and heuristic studies to guarantee the applicability of the findings as well as the ability to cope with real world settings. These three directions map to finer tasks, achievable within 12 months by a Ph. D. student or a postdoc.

SAIDA groups researchers from very different communities, overall covering a wide spectrum of expertise that traditional focused communities do not possess. SAIDA includes mathematicians with a very solid background in statistics, in probabilities, in rare event detection. SAIDA includes experts in high dimensional space where all data lie and where frontiers exist between classes, where neighbourhoods are established and where distributions, outliers and manifolds are central problems. SAIDA also includes experts from IFS with roots in signal processing, in human perception, in traitor tracing and trojaning, all this being mandatory to address trust, robustness and resilience of IA systems for defense. Naturally, SAIDA includes machine learning experts as well as specialists from the processing of multimodal data in order to cover a large range of tasks, from classification to retrieval e.g., overall involving large and complex heterogeneous data collections.