Physical and Intrinsic Security of Embedded Neural Networks – PICTURE

PICTURE is structured through three research axis. First, the PICTURE consortium aims at proposing an in-depth state-of-the-art on the threats targeting embedded neural network models, more especially software implementations. This compulsory work enables to define precise threat models relying on the adversarial knowledge (white/black box paradigms) and capacities (e.g., access to a clone device). Simultaneously, a set of use-cases is defined gathering public benchmarks and a set of hardware platforms (mainly, 32-bit microcontrollers). A focus is set on face recognition systems since these systems raise major integrity and confidentiality concerns.
Second, we aims at characterizing the critical properties of attacks using both algorithmic and physical means such as side-channel or fault injection analysis. For these physical attacks, cutting-edge platforms from the Center of Microelectronic of Provence are used. Several attacks will bring high interest for the consortium. Integrity of a model at inference time may be threaten by adversarial perturbations on the model inputs and also on the parameters stored in memory as well as critical instruction of the inference program. Model extraction (reverse engineering) is also a major concern for the ML models deployment and the use of side-channel analysis may significantly help an adversary to extract or recover information on a model and steal the IP or the performance.
Finally, jointly with the attack analysis, we develop tools and methodologies to properly evaluate the robustness of models as well as new defense approaches for embedded models. This works encompasses the evaluation of well-known countermeasures already proposed in other contexts (e.g., protection of cryptographic modules) for which we need to evaluate their relevance and efficiency on neural network models.

Ongoing project...

Today, Security is an important topic in the ML community with several dedicated sessions, workshops and issues in major conferences and journals. At the same time, security conferences proposed sessions focused on A.I. and particularly Adversarial ML (e.g., USENIX Security Symposium, ACM CCS, IEEE S&P). However, in both industrial and academic worlds, this consideration is feeble and AI is still driven by pure performance requirements with security as an option. A major objective of PICTURE is to facilitate a shift in the way to consider ML models by putting security at the core of the development and deployment strategy and anticipate and influence future standardization and certification strategies. From a social point of view, the consortium build PICTURE as a way to increase the trust of A.I. applications to a large audience by analyzing with transparency the threats and by demonstrating how to efficiently counteract.
PICTURE plans several publications in international conferences as well as the proposition of public benchmarks to help the scientific community to perform further analysis and studies.

Rémi Bernhard, Pierre-Alain Moellic, Jean-Max Dutertre, Luring Transferable Adversarial Perturbations for Deep Neural Networks, In International Joint Conference on Neural Networks, IJCNN 2021.

Mathieu Dumont, Pierre-Alain Moellic, Raphael Viera, Jean-Max Dutertre, Rémi Bernhard, An Overview of Laser Injection against Embedded Neural Network Models, In 7th World Forum on Internet on Things (WF-IOT), 2021.

Raphael Joud, Pierre-Alain Moellic, Rémi Bernard, Jean-Baptiste Rigaud, A Review of Confidentiality Threats Against Embedded Neural Network Models, In 7th World Forum on Internet on Things (WF-IOT), 2021.

K. Hector, P-A. Moellic, M. Dumont, J-M. Dutertre, A Closer Look at Evaluating the Bit-Flip Attack Against Deep Neural Networks, To Appear In 28th IEEE International Symposium on On-Line Testing and Robust System Design, IOLTS 2022, 12-14 September, Torino.

A major trend in Artificial Intelligence is the deployment of Machine Learning models even for highly constrained platforms such as low power 32-bit microcontrollers. However, the security of embedded Machine Learning systems is one of the most important issues to this massive deployment, more particularly for deep neural network-based systems. The difficulty comes from a complex twofold attack surface. First of all, an impressive amount of works demonstrate algorithmic flaws targeting the model’s integrity (e.g., adversarial examples) or the confidentiality and privacy of data and models (e.g., membership inference, model inversion). However, few works take into consideration the specificities of embedded models (e.g. quantization, pruning). Second, physical attacks (side-channel and fault injection analysis) represent upcoming and highly critical threats. Today, these two types of threats are considered separately. For the first time, the PICTURE project proposes to jointly analyze the algorithmic and physical threats in order to develop protection schemes bridging these two worlds and to promote a set of good practices enabling the design, development and deployment of more robust models.
PICTURE gathers CEA Tech (LETI) and Ecole des Mines de Saint-Etienne (MSE, Centre de Microélectronique de Provence) as academic partners and IDEMIA and STMicroelectronics as industrial partners that will bring real, complete and critical use cases more particularly focused on Facial Recognition.
To achieve its objectives, the consortium of PICTURE will precisely describe the different threat models targeting the integrity and the confidentiality of software implementation of neural network models on hardware targets from 32-bit microcontrollers (Cortex-M), dual architecture with Cortex-M and Cortex-A platforms to GPU platforms dedicated to embedded systems. Then, PICTURE aims at demonstrating and analyzing – for the first time – complex attacks combining algorithmic and physical attacks. On one hand, for integrity-based threats (i.e. fooling the prediction of a model) by combining principle of adversarial examples attacks and fault injection approaches. On the other hand, by studying the impact of the exploitation of side-channel leakages (side-channel analysis), even fault injection analysis associated to theoretical approaches to reverse engineer a model (model inversion) or to extract training data (membership inference attack). The development of new protection schemes will be achieved by the analysis of the relevance of state-of-the-art countermeasures against physical attacks (such an analysis has never been achieved at this scale). PICTURE will propose protections that will take place at different position within the traditional Machine Learning pipeline and more particularly training-based approaches that enable more robust models. Finally, PICTURE will present new evaluation methods to promote PICTURE results to academic and industrial actors. PICTURE aims at facilitating a shift in the way to consider ML models by putting security at the core of the development and deployment strategy and anticipate as well as influence future certification strategies.