Interpretability of Deep Neural Networks for Radiomics – INFORM

Deep neural networks (DNNs) have achieved outstanding performance and broad implementation in tasks such as classification, denoising, segmentation and image synthesis, including in medical imaging. However, DNN-based models and algorithms have seen limited adaptation and development within the radiomics approach, which aims at improving diagnosis or prognosis through extraction of engineered image features (intensity, shape, textures) sometimes combined with other clinical expert-derived features. We hypothesize that, despite the potential of DNNs to improve oncological classification performances in radiomics, a lack of interpretability of such models prevents their broad utilization, performance, and generalizability. The INFORM consortium thus proposes to investigate explainable artificial intelligence (XAI) with a dual aim of i) building high performance DNN-based classifiers and ii) developing novel interpretability techniques for radiomics. First, in order to overcome the limited amount of data typically available in radiomics, we will investigate Monte Carlo simulations combined with generative adversarial networks (GAN) for producing large amounts of highly realistic simulated images to facilitate training DNNs. Second, we tackle the interpretability of DNN-based feature engineering and latent variable modeling with innovative developments of saliency maps and related approaches for relevance scores. Both supervised and unsupervised learning will be used to generate features, which can be interpreted in terms of input voxels, conventionally engineered, and expert-derived features. Third, we propose to build explainable AI models that incorporate both conventional radiomic and DNN-based features. By quantitatively understanding the interplay between expert-derived and DNN-based features, our models will be easier to understand and to translate into clinical use. Fourth, preliminary evaluation will be carried out with the help of clinical collaborators on predicting outcome of patients with lung, cervical and rectal cancer. These proposed DNN models, specifically developed to reveal their innerworkings, will leverage the robustness and trustworthiness of expert-derived features that medical practitioners are familiar with, while providing quantitative and visual feedback. Overall, our methodological research and clinical application will advance interpretability of feature engineering, generative models, and DNN classifiers with applications in radiomics and broad medical imaging.
INFORM aims at maximizing the impact on the patient management of ML and DL techniques by developing novel methods to facilitate training of decision-aid systems for clinical treatment strategies optimization. The methodological approaches we propose in this specific area will play a major role in facilitating the acceptability of DL-based decision-aid systems relying on medical imaging for oncology. The proposed validated predictive models in various cancer types within the context of this project might subsequently be used to drive future prospective clinical studies in which patients could be offered alternative treatment strategies based on the results of these predictive models. Such a clinical and social potential is further enhanced by the public-private collaboration proposed in this project, where the developed methodologies will find their way in products.
The multidisciplinarity of INFORM is key to meet the target challenges and achieve the proposed goals. All partners have their individual world-leading qualifications and additional scientific expertise providing all the prerequisites for the efficient implementation of INFORM’s approach. The successful implementation of this project will have a large and prolonged impact both in the Medical/Oncology and the Computing/Artificial Intelligence field of predictive radiomics, as well as the same methodology could be extended to other diagnostic and therapeutic medical applications.