Biomimetic control of prosthesis 2: Operationalization in virtual and augmented reality, and on a real prototype – CoBioPro2

Upper-limb amputation is a profound trauma for both wounded soldiers and civilians, with significant consequences for the quality of life of those who undergo it. Despite major advances in upper-limb prosthetics, a high rejection rate persists, primarily due to the inadequacy of their control systems and the lack of necessary sensory feedback for their integration.

While prosthetics are still mainly controlled using the activity of only two muscles, and research in the field largely focuses on myoelectric control, we, in a previous project, explored an innovative control based on residual stump movements and knowledge of the movement goal. An artificial intelligence trained on a database of natural movements is used to reconstruct missing joints in individuals amputated above the elbow, enabling them to reach objects in virtual reality as easily as with a natural arm.

Several challenges, however, need to be addressed for this control to be applied in real-world applications and attract interest from industry professionals. This project aims to address challenges associated with two main but complementary research axes. The first axis aims to operationalize the control itself, providing proof of concept in the real world. The second axis, capitalizing on our excellent results obtained in virtual reality, aims to re-engage natural sensorimotor circuits for rehabilitation purposes.

The primary challenge associated with the first research axis concerns the determination of the movement goal. While this could theoretically be determined using tools from the rapidly advancing field of computer vision, it remains challenging to implement in practice and is ineffective for manipulating or moving already grasped objects. In this project, we will explore an alternative using gaze position and head orientation to determine the target posture of the prosthetic hand, which we will combine with our new prosthetic control before testing it on a real prototype.

The main challenge associated with the second research axis concerns the lack of proprioceptive sensory feedback. A vibratory sensory feedback congruent with object grasping will be implemented alongside our new prosthetic control to re-engage natural sensorimotor circuits in virtual reality. An extensive experimental program will then be conducted to develop and test the system's ability to re-engage natural sensorimotor circuits, to prevent psychophysiological effects of short-term or long-term immobilization, and its potential for treating phantom limb pain.

Beyond the specific applications targeted in this project, it holds broad prospects in the field of assistive and teleoperation robotics, as well as in rehabilitation, whether to compensate motor deficits related to strokes or to any form of immobilization.