Fusion of imaging and text data for longitudinal stroke prediction and intervention – IMAGE-TEXT-AVC

This project aims to improve prediction and intervention in strokes by integrating medical imaging data and textual reports. It is based on a robust database, RELATE, which collects clinical and imaging information from patients treated for ischemic stroke at the CHU de Lyon. Since 2020, this database has gathered data on over 1500 patients, allowing for detailed analysis of treatments and patient outcomes.
The main innovation lies in the fusion of multimodal data (images and text) and the use of advanced artificial intelligence models, such as Transformers. This will enable capturing the complex relationships between different types of data and modeling the longitudinal trajectories of patients. By incorporating patient evolution over time, the project proposes a model capable of predicting clinical outcomes while assessing the impact of interventions, such as thrombectomy and intravenous thrombolysis.
The project is divided into several components, including automated extraction of clinical information, data fusion, and analysis of causal relationships between interventions and clinical outcomes. The predictive model developed will be validated on the RELATE database and generalized to other hospitals. It will advance personalized stroke management by combining innovative clinical, ethical, and technical approaches.
The MYRIAD team at CREATIS and its partners, including experts in machine learning and MRI, will work together to accomplish this ambitious project, aiming to transform stroke care in France and beyond.