Characterization of cerebral and oculometric markers of the out-of-the-loop phenomenon during the supervision of an automated system in aeronautic context – EMOOL

In recent decades, the world around us has experienced profound technological changes, driven in particular by advances in autonomy, robotics and artificial intelligence. Today, human agents are regularly confronted highly automated, even autonomous systems, whose the primary role is to supervise these new artificial partners. This major change in the role assigned to human operators has generated new risks related to human factors. These riks include in particular difficulties in understanding artificial agents, detecting their errors and taking them in hand when necessary, a set of problems grouped together under the term "out of the loop phenomenon" (or ("OOL performance problem". Although widely studied, this out of loop phenomenon remains to this day very difficult to characterize, and even more difficult to compensate. Our observation is that of an explanatory weakness regarding the concepts and tools used when it comes to studying this phenomenon.In this context, our project aims to propose new tools inspired by recent work in cognitive and computational neuroscience in order to better understand and characterize this OOL phenomenon. and prevent it. In particular, through the use and joint analysis of electroencephalographic and oculometric signals, the EMOOL project aims to provide a tool to better characterize the neurocognitive mechanisms underlying the supervision activity of an automated system, but also to better understand how this activity evolves during OOL situations. One of the major and innovative interests of this coupling lies in analysis of the dynamics of the supervision activity during more ecological dynamic supervision tasks such as those envisaged in the research project in an aeronautical context (i.e. supervision task of obstacle avoidance assistance system), involving visual exploration and continuous information processing. The emotional context, so important in the aeronautical context, likely to modulate this visual exploration will, moreover, be modulated in order to study its influence on the supervision activity and the emergence of the OOL phenomenon. Finally, our project aims to develop a first model for estimating supervision activity based on brain and/or eye tracking markers using deep learning methods and methodological approaches taking into account the spatio-temporal dynamics of the OOL phenomenon. The scientific challenges are multiple, such as the induction of the OOL phenomenon, the manipulation of the emotional context, the combined analysis of EEG and eye tracking data in an ecological and dynamic context, and the identification of sufficiently discriminating and robust features of the OOL state in order to develop a model for estimating the supervision activity. The works proposed in this project, although instantiated here in tasks specific to aeronautics, seems to us both ambitious from a scientific point of view and potentially disruptive from an applied point of view. Finally, this work will open up new perspectives on the real-time compensation of this phenomenon.