Formal Analysis and Design of AI-intensive Cyber-Physical Systems – CyphAI

Artificial intelligence (AI) are revolutionizing information systems used for control and supervision of various devices (such as sensors, robots, IoT devices). Among such information systems are Cyber-Physical Systems (CPS) from which emerges a new generation of AI-intensive Cyber-Physical Systems that we call AI-CPS. Autonomous vehicles are examples to illustrate the synergy between CPS and AI, where the low-level engine control is typically a CPS, and the high-level control loop uses AI components t in sensors and scene recognition modules which are trained to learn how to classify road conditions and react to them. AI-CPS pose a number of design challenges. AI techniques are “unpredictable” due to a lack of formal framework to provide safety guarantees. In addition the existing CPS design methods rely on fixed models, while AI-CPS are supposed to learn from interactions with the environment and adapt their behaviors accordingly. It is imperative to ensure that their learning-enabled components work correctly because they may directly affect people’s lives and fortune. Self-driving car accidents caused by AI failures are striking real examples. In general, the outcomes of learning activities in AI components are not easily interpretable. When coupling CPS with AI, the increased heterogeneity in dynamics and behaviors can aggravate the reliability and explainability issues, if the learning activities are not properly formalized.
It is important to note that the existing frameworks for formalizing learning activities were developed for purely discrete or continuous settings, and extensions to the hybrid dynamics of CPS are scarce and ad-hoc. Together with the new design challenges, the combination of CPS and AI also opens new technological possibilities. Indeed one can benefit from the progress in AI to enhance the current CPS design approaches by combining model-based and data-driven approaches. Model-based design has been so far an efficient approach for CPS, by exploiting computing technology to create complex mathematical models to automatically analyze and implement these systems. Nevertheless, many phenomena emerging in the interaction between cyber-physical and AI components are not amenable to first principle analysis and discovering their dynamics should rely on data. However, data-driven models cannot provide causal mechanisms. The advantages and drawbacks of the model-based (white-box) and data-driven (black-box) approaches should be combined, leading to a gray-box approach that will be developed in this project. In addition, AI techniques can be used to derive efficient heuristics which are guaranteed to be safe if developed within a formal framework.
The research will be organized in 5 work packages that cover major design problems: WP1 Learning for CPS, WP2 Learning within CPS, WP3-Validation of CPS and AI-CPS, WP4 Monitoring and Control for Enforcing Dependability and Performance constraints, WP5 Case Studies, Applications and Tools. To achieve these objectives, we will combine formal methods with control theory, and we will use several tools from the field of functional analysis, differential equations, optimization, probability and statistics to solve our problems and establish mathematical rigor in our results. In particular we will develop mathematical concepts for measuring and sampling sets of AI-CPS behaviors, with respect to quantitative criteria. These concepts are necessary for formal reasoning and extracting information from data, to learn hybrid processes and to develop a gray-box approach for validation, control, and online monitoring. By providing an integrated design methodology and a computational platform for AI-CPS, our project is in line with the research direction pointed out in the Call “(2) The theory and technology for the refinement of mathematical models and the efficient application of simulations based on the collaboration of the model- and data driven types.”