Apprentissage automatique et imagerie multimodale pour la prédiction de la gonarthrose – MIMOSA

We investigated several deep learning methods for the early detection of knee osteoarthritis, using X-ray and MRI images.

In one approach using standard X-rays, a Siamese-based network was developed to analyse the medial and lateral compartments of the knee simultaneously, allowing the model to learn features that reflect symmetry, as considered in clinical assessments. Multiple layers were used to extract features at different levels, which were combined to improve performance. Training included a confidence-driven strategy that applied different loss functions to samples of varying certainty. The method was validated against assessments from experienced radiologists and extended to a broader classification task, demonstrating increased robustness and diagnostic reliability.

Another X-ray-based approach used a transformer-based framework to focus attention on clinically relevant regions, with a data augmentation technique that swapped key patches between images to improve model generalisation. Training employed a combined loss function, and cross-dataset evaluation confirmed reliable early detection. A third X-ray study introduced a generative model to create patient-specific three-dimensional knee images from X-rays. The model progressively reconstructed slices guided by anatomical depth and patient-specific features, generating full three-dimensional volumes. Additional inference steps allowed interpolation between slices, improving anatomical fidelity and demonstrating the potential for enhanced structural assessment.

Using MRI, one study proposed a three-dimensional residual convolutional network with integrated feature pooling and skip connections to learn volumetric representations. Clinically relevant structural markers, including bone growths and cartilage loss, were used to guide assessment. A semi-supervised learning approach leveraged both labelled and unlabelled data, while multi-view analysis exploited complementary perspectives to improve accuracy. Interpretability was supported by visualising regions influencing predictions.

Another MRI study introduced a network treating each three-dimensional scan as a set of two-dimensional slices and simultaneously analysing multiple views to capture comprehensive anatomical information. A specialised feature extractor was used to robustly identify discriminative features. Evaluated on a large dataset, the method outperformed single-view approaches, providing accurate early diagnosis. A further approach combined multiple MRI views with standard X-ray images in a multitask network capable of simultaneously predicting cartilage loss and meniscus tears. Features from different modalities were aggregated through a dedicated fusion strategy, and automated detection of knee regions streamlined feature extraction. Evaluation on a large cohort demonstrated robust and generalisable performance, supporting accurate, and relevant detection of knee osteoarthritis.

This project has developed advanced artificial intelligence methods to improve the early detection of knee osteoarthritis, utilising standard X-rays and magnetic resonance imaging (MRI) scans. These methods include models that analyse both sides of the knee simultaneously, focus on clinically important regions and generate three-dimensional images of the knee from standard two-dimensional X-rays. By using multiple views and combining information from X-rays and magnetic resonance scans, the systems can detect cartilage damage and meniscus tears more accurately and reliably.

A patent has been filed for the method of synthesising three-dimensional images from X-rays.

In collaboration with Medimaps, the project has delivered a fully functional computer-aided diagnosis system. This system integrates the developed models into a practical tool for clinical use and is supported by a secure online platform for testing and evaluation.

These advances represent a significant step towards an earlier, more precise and clinically relevant diagnosis of knee osteoarthritis, bridging the gap between research and real-world application.

This project has made significant advances in the early detection and assessment of knee osteoarthritis by applying novel artificial intelligence methods to standard X-rays and magnetic resonance images. The resulting applications include accurate and earlier diagnosis, enabling to identify subtle structural changes and intervene sooner. The system can simultaneously assess cartilage, meniscus, and other joint structures by combining multiple imaging views and modalities, providing a comprehensive evaluation of knee osteoarthritis. A key innovation is the ability to generate three-dimensional knee images from standard two-dimensional X-rays, providing detailed structural information without the need for further imaging, costs, or patient exposure. These capabilities have been integrated into a computer-aided diagnosis system developed with Medimaps to support automated evaluations and interpretable visual outputs. The creation of a platform also allows for the testing and performance benchmarking of new model components, as well as their potential integration into clinical workflows.

Several original avenues of research and development emerged beyond the initial project plan. For example, performing multitask learning to predict cartilage and meniscus damage simultaneously proved highly effective. Creating patient-specific three-dimensional images from X-rays using a generative approach represents a novel direction for structural assessment and diagnostic enrichment. Multi-view and multi-instance modelling strategies for magnetic resonance scans improved accuracy by capturing complementary anatomical information. Furthermore, the development of a fully functional demonstrator and the filing of a patent for the synthetic imaging method have highlighted unforeseen translational and commercial opportunities.

Overall, the project strengthened technical capabilities for the early detection of knee osteoarthritis, opening new avenues for clinical application, translational research and potential commercialisation. It also bridged the gap between innovative research and real-world healthcare solutions.

MIMOSA concerne l’arthrose du genou (gonarthrose) considérée comme priorité de santé publique. our la gonarthrose, le diagnostic est principalement évalué par classement visuel de la largeur de l'interligne articulaire et des ostéophytes sur des radiographies X. Cette évaluation est souvent inexacte et insensible aux modifications précoces car les changements sont faibles et le diagnostic nécessite des observateurs expérimentés pour être reproductible. Pour le pronostic et le diagnostic précoce de la gonarthrose, il est urgent de développer des méthodes permettant un diagnostic fiable sur les radiographies. La question est de savoir comment prédire l'apparition et l'évolution de la gonarthrose à l'aide de clichés radiographiques, afin d'améliorer et de reproduire son diagnostic voir sa prédiction ? Pour ce faire, l’apprentissage automatique ainsi que différentes modalités d’imagerie seront combinés. L’objectif est d’introduire un nouveau CADx transparent pour évaluer automatiquement la sévérité de la gonarthrose.