Adaptation of the automation strategy of autonomous vehicles (levels 3-4) to driver needs and driver state under real conditions – AutoConduct

The project is based on a multiple approach exploiting data about:
[1] human factors and ergonomics needs (User-Centered Design),
[2] the acceptance (a priori acceptability and use acceptance) to guide the developments of the autonomous vehicle which, until now, have been developed mainly according to technological criteria. The approach is based on the implementation of large-scale questionnaires and focus groups,
[3] the driver's state through objective measurements of indicators on three dimensions: physical, perceptive and internal (attentional and emotional) in driving simulator experiments,
[4] a progressive mode of shared control based on a physiologically valid model of sensorimotor control,
[5] the development of an adaptive Human-Machine Interaction,
The overall results of this project will be integrated on two instrumented vehicles. A wizard of Oz vehicle (active orders managed by a professional driver without the driver realizing it) will integrate the aspects «driver state / informative interaction modalities« to test the acceptability of the interfaces and the diagnoses robustness in open road driving condition. A second vehicle will integrate the entire system, including active controls on longitudinal and lateral vehicle dynamics. I will allow us to test the acceptability and the robustness of the Human-Machine Cooperation in a protected situation (on-track test ). The results of these assessments will lead to a safer and more efficient design of the 3-4 Level Autonomous Vehicle and will guide training programs for the autonomous vehicles driving.

The first phase (December 2016-June 2018) has consisted in defining and making consistent the different states of the art, the use cases, the needs analyzes as well as the specifications of the diagnoses and strategies of human-machine cooperation.
[1] A needs analysis was conducted through questionnaires on 2619 people and 6 focus groups. The results showed that the higher their acceptability of automated vehicles is, the more they choose high levels of automation. This choice is mainly important when individuals project themselves in a motorway driving condition rather than in a urban driving condition.
[2] An analysis of the driver profiles, including age and driving experience, was conducted and different design proposals for Human-Machine Interactions for the final experimental vehicles were developed based on driver preferences.
[3] Another objective being to create algorithms about postural diagnoses, visual strategies and internal states, specific data were collected in dedicated simulator experiments. In addition, another part of the work has focused on a new concept of shared control of the steering wheel with the aim of developing an evolutionary control law adapted to the problem of transitions between manual and automated controls and based on a cybernetic model of the driver.
[4] In parallel, the specifications of the driving strategies and the principles of Human-Machine Cooperation were highlighted in order to define both the criteria and the methods of transitions between automated and manual modes taking into account the driver's monitoring. Finally, an architecture and the choice of an integration platform for all the data have been specified.

At mid-term, the needs analysis helped feed the recommendations and the implementation of the HMI. This production has a significant potential for valuation in terms of publications. The specifications of the expected diagnoses have been formalized and the captors have been installed in the simulators for future studies. Initial results on shared control are promising and could allow direct application in vehicle design. The second phase will aim at collecting the diagnoses data, fusionning and integrating them into the experimental vehicles for the final experiments. The driver monitoring, resulting from the fusion of the diagnoses, will be tested under simulated driving conditions and under open road driving conditions, by integrating a HMI adapted to the diagnosed state of the driver in real time on the basis of a global indicator, which is one of the major challenges of the project with high potential of value.

The first phase has given rise to scientific productions: journals, international conferences and technical report.
Sentouh et al. (accepted), Driver-Automation Cooperation Oriented Approach for Shared Control [...], IEEE Transactions on Control Systems Technology.
Bel & Kraiem (2018). From Autonomous Driving Acceptability to AVs' Functions Acceptability [...]. ICAP, Montréal.
Pauzié & Ferhat (2018) Human-centred design recommendations for automatised car [...]. Humanist Conference, The Hague.
Béquet et al. (2018). Driver cognitive workload estimation [...]. DDI Conference, Gothenburg.
Hidalgo-Muñoz et al. (2018). Determination of cognitive workload variation in driving [...]. International Neuroergonomics Conference. Philadelphia.
Zhao et al. (2018). In Vehicle Driver Postural Monitoring using a Depth Camera Kinect, SAE Technical Paper.
Nguyen et al. (2018). Input-Constrained LPV Output Feedback Control [...]. American Control Conference, Milwaukee.

Driving automation, either level 3 (including unplanned manual control recovery) or level 4 (fully automated driving on specific road sections), introduces new safety and acceptability issues. In order to improve road safety by integrating advanced safety technologies, we must ensure that this new technology takes into account the needs and expectations of the drivers, on the one hand, and the predicable changes in user behavior, on the other hand. A Human Factors working group, bringing together the main French actors of driving automation, has been set up within the NFI plan (investment plan for the future) “autonomous driving” (http://www.economie.gouv.fr/files/files/PDF /nouvelle-france- industrielle-sept-2014.pdf) to prioritize these issues.

AutoConduct project aims to design a new Human-Machin Interaction (HMI) based on needs analyses and adapted to the driver’s condition in response to priorities identified by this working group. For this purpose, the current project will offer advanced monitoring of the driver by combining different diagnoses (physical state defined by the posture, internal states defined by emotions and cognitive load, and perceptive state defined by visual strategies) to adapt the management of interactions between the driver and the vehicle automation in real time.

Use cases examined in the project will investigate transitions from manual to automated driving, and vice versa as well as transitions from automated driving to manual. HMI management and acceptability in specific use cases (e.g. safety maneuvering process initiated by the system) will also be investigated. The advanced driver monitoring system and the interface developed by the project will be integrated in two instrumented vehicles.

A wizard of Oz instrumented vehicle (i.e. the active controls are managed by a professional driver hidden from the driver’s view) will integrate driver monitoring and feedback interaction to test HMI acceptability and robustness of diagnostics on public roads. A second instrumented vehicle will integrate the entire system, including active controls of the speed and direction of the vehicle, to test the acceptability and robustness of HMI in a controlled environment (i.e. test track). The results of these evaluations will contribute to improve the safety and the effectiveness of the design of automated vehicle as well as providing inputs and recommendations for driver training on automated driving.

The originality of this project is to (1) adopt a user-centric approach based on the needs of human factors and ergonomics (2) take into account acceptability (a priori and after experience with the system) at the early, design stages of automated vehicle, which, until now, were developed primarily based on technological criterion, (3) treat the driver’s state through objective measures of indicators on three dimensions: physical, perceptual, and internal (attentional and emotional), (4) develop a progressively shared vehicle control (i.e. cooperation) based on a physiologically valid sensorimotor control model. This type of user-centric approach, already adopted by the Humanist Network of Excellence "Human centered design for ITS", has proven to be effective for other applications dedicated to the driver in the development of prototypes. It will, thus, consist in applying a user-centered approach to the field of automated driving and developing validation methodologies of adaptive strategies in naturalistic conditions.

The current project is following ABV, PARTAGE, and CocoVeA projects (focusing on level 2 of automation) and capitalizes the results of the European project ACROSS in the field of pilot monitoring applied to aeronautics.