Continuous Monitoring and Improvement for Trustworthy AI: A Case Study on content classifiers – WestOps

Ouest France, a leading media company, uses Machine Learning (ML) techniques for content classification, information extraction, and recommendation. Ouest-France is exploring new approaches for MLOps implementation to streamline the evaluation process, ensure timely detection of performance variations, and enable faster and more reliable adaptation of ML components.
The WestOps project aims to address these challenges. Firstly, it aims to simplify integrating functional non-regression tests for classifiers and recommender systems. This will be achieved through a custom domain-specific language that seamlessly integrates within Ouest France's existing ML system. Secondly, the project focuses on early detection of model deviations using variability techniques that incorporate journalist validation. By analyzing results from various models, the project aims to identify and address performance issues. Furthermore, a monitoring module will be developed to raise alerts and trigger feedback loops, further improving the maturity of the MLOps pipeline.
The expected outcomes of WestOps include a global MLOps framework validated through the Ouest France case study, an open-source unified test framework for ML models, and the implementation of performance monitoring and feedback loops in a real-world setting.
The collaboration between Ouest-France, Kereval, and Inria DiverSE presents a groundbreaking opportunity to strengthen the reliability of AI systems for industrial partners. This project's focus is the development of modular MLOps components, designed for adaptation across diverse sectors. By promoting this adaptability, the project will stimulate significant improvements in global machine learning practices and applications.