Machine Learning for Multimodal Medical Image Reconstruction – MultiRecon

Multimodal medical imaging (e.g., PET/CT, PET/MRI) plays an important role for diagnosis and research. Multimodal machine learning (MML) aims at develop methods that can process information from multimodal imaging, thus allowing “to learn” the dependencies between the modalities. Image reconstruction with MML can take advantage of the dependencies between the images and reduce the image noise, thus allowing for patient dose reduction while preserving image quality. However, conventional dictionary learning techniques are memory consuming and cannot be applied to multimodal image reconstruction. The objective of the project MultiRecon are: (i) to develop new mathematical techniques for less memory-consuming convolutional dictionary learning (CDL) techniques for multimodal image joint reconstruction in PET/CT and PET/MRI, (ii) to further extend these methodologies for dynamic imaging (motion estimation/compensation and kinetics), and (iii) to disseminate our research by implementing the developed methodologies on the open-source reconstruction platform CASToR.