Virtual humanoids learning motion skills
In a nutshell, the project aims at exploring the use of learning algorithms in the context of humanoid motion to construct repertoires of skills that can be mobilized adequately to create complex motions. For that purpose, the focus is put on the exploration of specificities and techniques related to humanoid motion than can be incorporated into learning algorithms.
Instead of following a benchmark approach that would consider humanoid motion as just another example of problem that can be tackled by reinforcement learning, we propose to concentrate our work on the particularities of humanoid motion, and on how specialized learning algorithms can take advantage of them. The project focuses in particular on three aspects of reinforcement learning: exploration, representation learning and modularity.
At the end of the project, the developed algorithms will be made available in an open-source tool.
The main perspective of the scientific results of the project is to advance the fields of reinforcement learning, humanoid robotics and virtual character animation.
1. Thomas Pierrot, Guillaume Ligner, Scott Reed, Olivier Sigaud, Nicolas Perrin, Alexandre Laterre, David Kas, Karim Beguir, Nando De Freitas, Learning Compositional Neural Programs with Recursive Tree Search and Planning, NeurIPS 2019.
2. Astrid Merckling, Alexandre Coninx, Loic Cressot, Stephane Doncieux, Nicolas Perrin, State Representation Learning from Demonstration, LOD 2020.
3. Matheron et al., Understanding Failures of Deterministic Actor-Critic with Continuous Action Spaces and Sparse Rewards, ICANN 2020.
4. Guillaume Matheron, Nicolas Perrin, Olivier Sigaud, PBCS: Efficient Exploration and Exploitation Using a Synergy Between Reinforcement Learning and Motion Planning, ICANN 2020.
5. Alexandre Chenu, Nicolas Perrin-Gilbert, Stéphane Doncieux, Olivier Sigaud, Selection-Expansion: A Unifying Framework for Motion-Planning and Diversity Search Algorithms, ICANN 2021.
6. Thomas Pierrot, Nicolas Perrin-Gilbert, Olivier Sigaud, First-Order and Second-Order Variants of the Gradient Descent in a Unified Framework, ICANN 2021.
Human motion has several particularities that make it difficult to reproduce with algorithms. It relies on numerous heuristics that combine well and are mobilized and adjusted with great flexibility. For the control of humanoid robots, powerful techniques exist, mainly based on links between classical mechanics, kinematics, linear algebra and optimization. Yet, these approaches (which we refer to as "model-based") do not permit to generate complex autonomous behaviors, because many components of the movements must be described by precise tasks. Recently, machine learning has lead to interesting results without exploiting model-based methods. The objective of this project is to go further by developing specific machine learning techniques that share some aspects of the model-based methods and permit to automatically construct repertoires of motion skills with virtual humanoid robots.
Monsieur Nicolas PERRIN (Institut des Systèmes Intelligents et Robotiques)
The author of this summary is the project coordinator, who is responsible for the content of this summary. The ANR declines any responsibility as for its contents.
ISIR Institut des Systèmes Intelligents et Robotiques
Help of the ANR 139,665 euros
Beginning and duration of the scientific project: - 48 Months