Hiérarchisation et Apprentissage par Renforcement Relationnel – HARRI

Reinforcement Learning considers systems involved in a sensor-motor loop (e.g., a robotic system perceiving its environment through sensors, and acting thanks to effectors). Instead of handcrafting the systems reactions in any possible situation, RL postulate is that the system should automatically acquire – through machine learning – adequate reactions. Classical RL techniques handle attribute value state representations, such vectors storing the distance of the robot with the nearest objects. Most research works in RL aims to discover efficient algorithms handling such propositional representations. In the HARRI proposal, we intend developing RL algorithms and specifically target algorithms that handle a more complex representation of states and actions. Assuming our target language for representing states and actions is a restriction of first order logic (refered hereafter as relational), situations are described in terms of predicates encoding relationships between objects of the environment, as opposed to numerical vectors for attribute-value representations. This paradigm shift opens new perspectives for reinforcement learning, in particular with respect to application scale-sup. Relational Reinforcement Learning (RRL) requires competence in both: - Reinforcement Learning for discovering new algorithms for making the most out of this new representation formalism - Inductive Logic Programming, a subfield of symbolic Machine Learning dedicated to the development of algorithms that learns concepts and find regularities expressed in restrictions of First Order Logic, for discovering new algorithms that cope with the constraints of reinforcement learning, i.e., incrementality and stability. The team involved in the project gathers young researchers of both fields – RL and ILP – all working in the same research laboratory (note that LIPN is the only lab in France that gathers specialists of both fields). The scientific goal of the HARRI proposal is to quickly invest this rapidly emerging field of RRL, thanks to the complementary expertises of members of the group. Application development will also be an important aspect of the proposal, tackling real domains such as video-games.