Artificial Intelligence in Real-Time systems – ARTi

The growing utilization of artificial intelligence (AI), like deep neural networks, in embedded real-time systems poses many challenges. One of these is the very large quantity of data processed by these algorithms, which impacts their execution time and power consumption on resource-limited platforms. To address this problem, software and hardware approaches are increasingly being adopted. On the one hand, hardware accelerators (e.g., FPGA, GPU, TPU) make it possible to optimize matrix calculation and process the calculation divided into several subtasks in parallel. On the other hand, software techniques (e.g., network pruning, and quantization) make it possible to reduce the size of the network.

This project seeks to explore the potential of hardware accelerators and software optimization techniques in achieving efficient and accurate AI inference in embedded real-time systems. Our approach involves developing new scheduling policies specifically designed for hardware accelerators, which can dynamically regulate the balance between time, energy, and accuracy. These policies require an offline optimization framework, which, in the iterative process, can determine the level of compression applied to the neural networks to ensure the schedulability of the resulting neural net tasks and their energy and accuracy indicators. We plan to evaluate the proposed framework on an autonomous F1TENTH car and mini rover with an Edge TPU accelerator.