Algorithms, Approximations, Sparsity and Sketching for AI – AllegroAssai

Solid algorithmic and mathematical foundations are essential to endow AI systems with guaranteed utility, resource-efficiency and trustworthiness. Exploiting massive data streams requires controlling the tradeoffs between performance and computational footprint. For example, sensors for autonomous vehicles generate terabytes of data per day per car. Another constraint is to comply with European regulation (GDPR) e.g. to ensure privacy when captured video streams feature pedestrians or licence plates. Similar constraints arise for AI systems that learn biomarkers from sensitive medical imaging data, with an even stronger emphasis on privacy-preservation, performance guarantees, and robustness in adversarial settings.
AllegroAssai will address the fundamental scientific challenge of designing AI techniques endowed not only with solid statistical guarantees (to ensure their performance, fairness, privacy, etc.) but also with provable resource-efficiency (e.g. in terms of bytes and flops, which impact energy consumption and hardware costs), robustness in adversarial conditions for secure performance, and ability to leverage domain-specific models and expert knowledge.
The vision of AllegroAssai is that the versatile notion of sparsity, together with sketching techniques using random features, are key in harnessing the fundamental tradeoffs of AI.
The first pillar of the project will be to investigate sparsely connected deep networks, to understand the tradeoffs between the approximation capacity of a network architecture (ResNet, U-net, etc.) and its “trainability” with provably-good algorithms. A major endeavor is to design efficient regularizers promoting sparsely connected networks with provable robustness in adversarial settings.
The second pillar revolves around the design and analysis of provably-good end-to-end sketching pipelines for versatile and resource-efficient large-scale learning, with controlled complexity driven by the structure of the data and that of the task rather than the dataset size.
To achieve its ambitious goals, AllegroAssai will leverage the modern avatars of the concept of sparsity and combine them with advanced high-dimensional function analysis, information geometry, sketching techniques for dimension reduction by distribution embeddings, and nonconvex optimization. This will be performed through a continuous feedback between theoretical investigations and empirical studies on targeted applications. The resulting computing pipelines will be implemented in software and using optical processing wherever possible to reduce the energy footprint, in an agile process. Software packages for distributed aggregation of sketches, sketched learning, and sparse network learning will be developed to ensure a wide dissemination of the results.
Thanks to the developed resource-efficient sketching and sparse network approaches, AllegroAssai will contribute to a more ecological, less-energy consuming AI economy favoring the convergence of the ecological transition and the development of AI. Memory-efficiency and privacy-preservation will allow sharing and aggregating of massive data volumes across actors of the transport industry, further opening the door to new ambitious data policies for transport.