Examining the Interplay of Neurocognitive Systems in Early Stages of Reading Acquisition – MULTI_READ

This project aims to explore how different neurocognitive systems interact with the visual system during reading acquisition, a process that involves mapping visual symbols (letters) to language sounds. Although reading acquisition is thus inherently multimodal, the exact role of other systems (speech production and writing systems) in this process is not well understood. The project focuses on how multimodal training impacts letter representations and letter-sound mapping, key components for reading.
We will conduct behavioral studies and a functional magnetic resonance imaging (fMRI) study on adults learning a new alphabet with pseudo-letters, as well as complementary studies with kindergarten children using real letters. Our investigation will examine the effects of various training methods with different levels of multimodality (i.e., more or less systems engaged) on visual representations of letters, including size, shape, and mirror invariance (the latter being the ability to recognize images from left and right perspectives), as well as how training impacts letter-sound correspondence and sleep consolidation.
The hypothesis is that increasing the number of cross-system mappings during training will create a more robust and flexible letter representation. This should enhance size and shape invariance, but inhibit mirror invariance (crucial for differentiating mirrored letters like "b" and "d"), and improve learning consolidation after sleep. However, the effectiveness of these outcomes is expected to depend on how different neurocognitive systems are engaged during training, as poorly coordinated engagements could reduce learning gains.
This research could advance theoretical understanding of multimodal cognition and learning and have practical implications for educational methods by providing insights into optimizing literacy training.