Intelligent Context-awareness for Increased Safety and Quality in Robot Assisted Surgery – i-SaferS

Robot-assisted surgery, allowing surgeons to perform complex surgeries through tiny incisions, has been significantly increasing in popularity worldwide. However, surgical safety is still a major concern in the high-risk operating environment and remains a threat to the quality of surgical outcome. As global statistics, millions of surgeries per year would encounter safety-critical intraoperative adverse events, most of which were otherwise avoidable if the surgeon can be timely aware of the potential risks in operation. In this project, we aim to introduce smart context-awareness into robot- assisted surgery, by developing novel artificial intelligence techniques to provide automatic cognitive assistance for surgeons during critical moments of the procedure, in order to improve surgical safety and quality. The use case of this project will be robot-assisted hysterectomy, which is the most common gynecological procedure performed on women diagnosed with uterine fibroids or cervical cancer. Both Hong Kong and French teams will explore together innovative multimodal machine learning methods, based on available synchronized clinical video and kinematic data, which will be more advanced and clinically relevant than all existing methods that only used visual perception. Based on our pilot studies, we have identified a set of critical intraoperative scenarios to address avoidable adverse events in hysterectomy, such as injury of the pelvic ureter during both the coagulation of the uterus pedicle and adnexectomy. To achieve our goal, we will solve the following key challenges: 1) How to yield precise and real-time recognition of the surgical context, i.e., surgical workflow, operation actions, surgical instruments, anatomical tissues and the reconstructed 3D surgical environments. 2) How to conduct automatic assessment of the identified critical-context-of-safety (CCS), and further provide informed decision-making support to surgeons for their best practice to avoid safety risks. By a research collaboration between world-class teams with complementary expertise and already-available clinical and annotated data, the i-SaferS project will generate outputs that provide fundamentally new and generic solutions and impactful references to the field. The project outcomes will significantly contribute to the emerging field of intelligent robotic surgery, and further strengthen the leading competitiveness of both partners in this field nationally and internationally.