Biomimetic control of prosthesis using residual movements and contextual information – CoBioPro

Upper limb amputation is a serious trauma for an injured soldier as for a civilian. This is a type of accident that is unfortunately frequent for soldiers in the field, but which is also encountered in civilians, with a total amount of upper limb amputated people in France estimated between 8000 and 15000. Technological developments now offer to patients the possibility to benefit from myoelectric prosthesis, which are typically controlled by the activity of two residual muscles, and are reimbursed by social security. However, currently available control systems for those prosthesis remain a major problem, which worsen as the level of amputation increases, since the control will handle a higher number of joints with fewer residual muscles. Together with the lack of sensory feedback, the complexity of existing prosthesis control systems is responsible for the high abandon rate associated with their use. This implies a deterioration of patients’ functional abilities, and highlight the failure of important human (medical doctors, rehabilitation therapists, prosthetists/orthotists…) and financial (material, research, salaries…) resources invested.

Whereas most researches dedicated to improve prosthesis controls focusses on myoelectric control based on residual muscle activities, a promising alternative aims at exploiting residual movements that are far more reliable and easier to interpret than muscle signals that are subject to noise and various artifacts. However, this approach faces a problem similar to that of myoelectric control with high level amputation, which implies more joints to control with less residual joints motion. This ASTRID project aims at unlocking this critical difficulty by making additional use of contextual information on the movement to produce, in order to enable natural and intuitive control of the multiple joints of prosthesis.

Our preliminary results in virtual reality proved the efficiency of our approach, which enables subjects to grasp bottles of various positions and orientations with performance similar to that with their true arm, whereas 4 distal joints were controlled on the basis of shoulder movement and contextual knowledge about the movement to produce. This project aims to build upon and develop this initial proof of concept on tasks that are more complex and relevant to amputees, as well as handle the necessary transition from virtual reality to a robotic platform enabling to operate and manipulate real objects, and to get closer to a real prosthesis. Whereas movements from elbow to wrist included will be handled by our novel control, the grasping movement of the hand will be handled by a simple myoelectric control with sensory feedback, developed in a DGA funded PhD recently conducted in the coordinating team.

More than a simple robotic arm, the test-setup developed in this project is designed to host the various tools from artificial intelligence involved in our control (automatic object recognition with computer vision, machine learning, encoding motor coordination with deep neural networks), as well as the various kinematic and physiological signals used (inertial measurement units, optical sensors, EMGs, ocular movements). In addition to its specific use in this project to test and improve prosthesis control systems, this setup offers a high potential for the development of human-machine interface based on various sources data fusion.