edge AI tranSformer-based adaptable human roboT collaboration system for smartER Industry 4.0X – Asterix

Facing a growing demand in industries for customised products and shortage of qualified labour, the Astérix project aims to develop a smart, safe, easily adaptable edge AI empowered transformer-based human robot collaboration (HRC) system for assembly tasks in industry as an effective solution to provide flexibility and intelligence in automation. For this purpose, the Astérix project proposes to: 1) make use of state of the art edge AI cameras to perceive and analyse the behaviours of human operators for better HRC efficiency with a decreased latency, lower energy consumption and enhanced data privacy; 2) leverage the foundation models empowered novel AI paradigm based on transformers for the design of the smart supervisor, human robot interactions, scene understanding, and robot control, thereby making the proposed HRC system general purpose and generalisable, providing easy use and control, adaptability and robustness with respect to various changing factors in dynamic environments, e.g., scenes, objects, tools, assembly plan; 3) design novel architecture optimization schemes to enable the inference of transformers on the edge, thereby improving the latency performance while saving both energy and memory; 4) train transformer-based models for robot control and its interactions with workers in harnessing large-scale simulated robot manipulation data thanks to the huge progress made in various robot simulation environments as well as human demonstrated data through a cutting edge robot teleoperation system with haptic feedback.

To make such a system possible, the Astérix project gathers from the two countries and along the technology value chain 4 world-class research groups with complementary expertise along with two key industry stakeholders: 1) the global company DENSO CORPORATION for HRC enabled assembly use cases; 2) Prof.Harada’s group at Osaka University (OU) with a longstanding experience on intelligent human robot collaboration; 3) Prof.Chen’s group at the Liris lab within Ecole Centrale de Lyon (ECL_Liris) with a confirmed expertise on computer vision, robot learning and simulation empowered self-supervised learning; 4) Prof.Bosio’s group at the INL lab within Ecole Centrale de Lyon (ECL_INL) with profound knowledge on DL compression techniques for edge devices and and heterogeneous architectures; 5) Prof.Hasegawa’s group at Nagoya University with an outstanding expertness on robot teleoperation systems and learning by demonstration; 6) Asygn, a French high tech SME specialised in sensors and close-to-sensor computing with growing experience in edge AI computing. They already know each other quite well through past projects or under way, conferences and joint courses.