Automatic Endoscopic Scene Assessment for Safety Checkpoint Validation in the Operating Room – AI4ORSafety

The project AI4ORSafety aims at proposing new computer vision and machine learning methods for the analysis of endoscopic videos so as to build an AI system for the operating room that can automatically monitor safety checkpoints. This project will focus on a high-impact and high-visibility clinical application that my research group has recently started investigating, namely automatically monitoring a safety step called the “critical view of safety” (CVS) during laparoscopic cholecystectomy procedures.

This CVS step is strongly recommended by international surgical societies to reduce the rate of bile duct injuries (BDI) during the laparoscopic cholecystectomy procedure. The same societies highlight the need for new tools to consistently enforce the CVS, as BDI in these procedures is still double the injury rate observed in open surgeries. Since this type of surgery is very common – over 1 million per year in the USA alone - enforcing the CVS safety manoeuver can have a strong socio-economic impact. We believe that AI and computer vision can help in this situation and already have partnerships with an international group of 10 clinicians, who have contributed to a feasibility study.

CVS consists of dissecting and visualizing the anatomy in a particular way, to ensure confidently that it is well dissected. We believe that such a task lends itself very well to automation and has strong potential for improving the safety of the procedure. Our objective is to ensure, in a consistent manner across hospitals over the world, that the step has been performed, since studies show that even when the CVS is reported to be performed, independent post-operative reviews of the videos show that it is not, by a large margin (18% overlap only). In the feasibility study, we designed with a team of clinical partners an assessment scheme suitable for AI automation that is based on several binary criteria to characterize whether the CVS is achieved or not. The data is however visually very challenging, especially due to the nature of the anatomy and to camera motions.

Considering these challenges, we propose new methods for (1) self-supervision on endoscopic videos, where classical self-supervision methods have limited success due to the nature of the data; (2) weakly-supervised anatomy segmentation on video clips, since the anatomy can not only help for CVS prediction, but also for the explanation of the decision; and (3) grading video clips and providing an understanding of the prediction, a much needed feature in clinical AI. Our key contribution will be to develop these methods for endoscopic videos, for which no solution exists. There is a high potential for using them to implement safety checks routinely for several kinds of surgery. We will also develop a real-time prototype for live demonstration in the OR. In the long term, we envision to use these AI techniques in a surgical control tower, akin to a control tower in aeronautics, that can exploit digital surgical data to monitor safety and progress during surgery. The IHU Strasbourg hospital, on the premises of which my research group is located, has built the facilities necessary for such a surgical control tower and is strongly committed to implementing it in the next few years.

We will demonstrate our results on a large database of endoscopic videos. We will start with an initial database of 600 videos that my group has already collected and collect 600 additional videos from our international partner institutions. This chair project has received strong support from the University of Strasbourg, the IHU Strasbourg hospital, and from an international team of clinicians. If successful, we believe that the AI-enforced safety checks could be deployed rapidly in multiple ORs thanks to our large clinical network and highlight internationally the potential for surgical AI research in France.