Deep Learning for Volumetric Brain Analysis: Towards BigData in Neuroscience – DeepVolBrain

Whole brain segmentation using deep learning (DL) is a very challenging task since the number of anatomical labels is very high compared to the number of available training images. To address this problem, previous DL methods proposed to use a single convolution neural network (CNN) or few independent CNNs. In this project, we present a novel ensemble method based on a large number of CNNs processing different overlapping brain areas. Inspired by parliamentary decision-making systems, we propose a framework called AssemblyNet, made of two «assemblies« of U-Nets. Such a parliamentary system is capable of dealing with complex decisions, unseen problem and reaching a consensus quickly. AssemblyNet introduces sharing of knowledge among neighboring U-Nets, an «amendment« procedure made by the second assembly at higher-resolution to refine the decision taken by the first one, and a final decision obtained by majority voting.

The main results of the project at T + 18 concern the following elements:

- Development of new semi-supervised learning methods for 3D MRI segmentation. This method is currently used for the segmentation of lesions in multiple sclerosis.

- Proposal of a new collaborative learning paradigm allowing the segmentation of a large number of structures. This new paradigm will be extended to multitasking and multiscale analysis.

- Proposal of a quantitative method for quality control of 3D registration of brain MRIs. This method is now used in all of our studies and has already demonstrated its effectiveness.

- Development new architecture for our web platform. This new version is already accessible in beta mode.

In the second part of this project, we will continue to study the use of a large ensemble of networks to solve complex tasks. Then we will apply our new methods to large databases to generate new knowledge for neuroscience and clinical research. We will also develop new software that will be integrated into our open access web platform.

[J1] P. Coupé, J. V. Manjon, E. Lanuza, G. Catheline. Lifespan changes of the human brain in Alzheimer's disease. Nature Scientific Report, 2019.
[J2] K. Hett, V.-T. Ta, G. Catheline, T. Tourdias, J. V Manjón, P. Coupé. Multimodal Hippocampal Subfield Grading For Alzheimer’s Disease Classification. Nature Scientific Report, 9 (1), 1-19, 2019.
[J3] J. V. Manjon, A Berto, J. E. Romero, E. Lanuza, R. Vivo-Hernando, F. Aparici-Robles, P. Coupé. pBrain: A novel pipeline for Parkinson related brain structure segmentation. Neuroimage Clinical, 2020.
[J4] P. Coupé, B. Mansencal, M. Clément, R. Giraud, B. Denis de Senneville, V.-T Ta, V. Lepetit, J. V. Manjon. AssemblyNet: A large ensemble of CNNs for 3D Whole Brain MRI Segmentation. Neuroimage 2020.
[J5] K. Hett, V-T. Ta, I. Oguz, J. V. Manjon, P. Coupé. Multi-scale Graph-based Grading for Alzheimer's Disease Prediction. Medical Image Analysis, 2020
[C1] P. Coupé, B. Mansencal, M. Clément, R. Giraud, B. Denis de Senneville, V.-T Ta, V. Lepetit, J. V. Manjon. AssemblyNet: A Novel Deep Decision-Making Process for Whole Brain MRI Segmentation. MICCAI'19, 2019.
[Preprint1] J. V. Manjon, P. Coupé. MRI denoising using Deep Learning and Non-local averaging. 2019, arXiv. arxiv.org/abs/1911.04798
[Preprint2] J. V. Manjon, J. E. Romero, R. Vivo-Hernando, G. Rubio-Navarro, M. De la Iglesia-Vaya, F. Aparici-Robles, P. Coupé. Deep ICE: A Deep learning approach for MRI Intracranial Cavity Extraction. 2020, arXiv. arxiv.org/abs/2001.05720
[Preprint3] J. V. Manjon, J. E. Romero, P. Coupé. DeepHIPS: A novel Deep Learning based Hippocampus Subfield Segmentation method, arXiv. arxiv.org/abs/2001.11789
[Preprint4] B. de Senneville, J. V. Manjon, P. Coupé. RegQCNET: Deep Quality Control for Image-to-template Brain MRI Registration, arXiv. arxiv.org/abs/2005.06835

Magnetic resonance (MR) imaging plays a crucial role in the detection of pathologies, the study of brain organization and the clinical research. Every day, a vast amount of data is produced in clinical settings and this number is increasing rapidly, which prevents the use of manual approaches for data analysis. As a result, the development of reliable segmentation techniques for the automatic extraction of anatomical structures is becoming an important field of quantitative MR analysis. In the DeepVolBrain project, the final goal is to develop a new generation of quantitative MRI analysis methods to cope with the rise of BigData in neuroimaging and, ultimately, to generate new knowledge. Moreover, the proposed methods will be implemented in open access to the entire community through a web platform.

Objective A: First, we propose to develop novel methods by addressing the current limitations of Deep Learning (DL) in neuroimaging. DL is a fast-growing field in computer vision that has recently obtained many successes. However, so far, results obtained by DL for MRI segmentation are not as good as expected. The limited performance of DL in neuroimaging seems resulting from several factors such as few training data or large memory requirement. First, we propose to address the problem related to the limited number of training data by increasing the size of training library and by reducing the number of required training images. To this end, we will develop new data augmentation strategy and innovative DL architectures enable to improve learning speed and to reduce the number of required training images. Second, to address the memory issue related to DL, we will propose ensemble learning strategy based on locally adaptive 3D CNN. Finally, the last factor limiting the performance of DL is the quality of preprocessing to compensate for the image heterogeneity. Based on our extensive expertise, we will integrate the developed DL segmentation methods into robust pipelines.

Objective B: The emergence of very large datasets opens up new challenges related to BigData as defined by the usual 3Vs model (Volume, Variety and Velocity). The fast and robust pipelines developed for Objective A will address these challenges by proposing new tools able to process large Volume of data, to cope with image Variety from different datasets and to propose high Velocity thanks to GPU-based computing. However, two Vs have been recently added to the usual 3Vs Big Data model – Veracity and Value. In medical imaging, the reliability is related to the questions of quality control (QC) and traceability. Therefore, to ensure Veracity of the produced results, we need to propose advanced QC and to estimate a confidence of the produced results. In the DeepVolBrain project, automatic QC and error correction will be integrated into pipelines to increase the confidence in the results produced. Moreover, an extensive validation over large scale datasets will be carried out. Finally, the proposed tools will be applied to large datasets including pathological cases to demonstrate the Value and the capability of the proposed project to produce new knowledge.

Objective C: Finally, we will make our tools freely available by deploying a web platform. In the past, with the volBrain platform, we developed an original platform in a fully open access hosted at Valencia in Spain. This platform already processed more than 75 000 MRI in 3 years. This unexpected very high number of online processing pushes us to investigate new strategies to make our platform more scalable and to ensure its sustainability. In this project, we will first achieve the deployment of a second site in France at the LaBRI. Second, we will propose a new scalable and flexible architecture. This new architecture will enforce the security and privacy of the data. Finally, each of developed tools will be integrated in this new platform.