The Artificial Motion Factory – ARTIFACT

Today’s robots are confined to tightly controlled environments, and even the complex choreographies that the Boston Dynamics humanoids flawlessly execute heavily rely on motion capture, handcrafted control strategies, and detailed workspace models, with little place for sensing. To put it bluntly, robots are nowhere near the level of agility, dexterity, and even less so autonomy, robustness and safety required for their deployment “in the wild” alongside people. A quantum leap in these capabilities is required for them to fulfill their promise and truly step out of the laboratory.
Our tenet is that the key to this revolution is the development of the theoretical and algorithmic foundations of a true artificial motion intelligence, an AI with the added challenge of physically interacting with dynamically changing environments and ultimately people. We will break away from the dichotomy between optimal control, where the role of perception is traditionally limited to an early state estimation stage, and reinforcement learning, where control policies are typically learned in a model-free fashion, with no guarantee to cope with the curse of dimensionality.
Concretely, we will use the Koopman model of complex dynamical systems to learn sensorimotor models and the corresponding control strategies from sensor data. We will develop powerful methods for learning, controlling and sharing a dictionary of sensorimotor synergies across tasks, echoing those used by the human central nervous system in everyday tasks, and speeding up the acquisition of new skills. We will leverage the compositionality of sensorimotor strategies and tree-search strategies powered by neural networks to optimally plan robot motions under dynamic observation constraints. The proposed framework will be implemented in new differentiable programming software architectures and demonstrated on several locomotion and manipulation tasks, both in simulation and on real robots.