Mental simulation for spatial memory and language comprehension: from verbal to virtual in children and adults – SiMuLang

Dual-task paradigms will be used and adapted according to participants (adults or children). In such paradigms, action-related sentences immediately precede the presentation of a related picture or a related motor response. Participants are typically asked to understand sentences describing motor events or object’s orientation and to perform a judgment task on the picture or a motor task designed to match or mismatch the meaning of the sentences. With this paradigm, a mismatch effect has been reported suggesting shared perceptual/motor processes between language meaning and action. These interference paradigms are very fruitful allowing the manipulation of various parameters. In addition, whereas interference has been so far mainly assessed at the response stage (for instance by subsequently presenting an image or asking for a motor response that matches or not with the preceding sentence), we will also assess it during the encoding stage (by concurrently stimulating motor programs, at the physical or neuronal level).
An important transverse dimension in this series of experiments will be the taking into account of individual differences, related to reading comprehension in children, and to spatial abilities in adults. Another dimension involved in the logic of experiments consists of comparisons between the verbal presentation of information (words or phrases) and modes of visual presentation (images or virtual environments). Varying the encoding mode - an innovative factor - should help isolate the impact of motor simulation by controlling part of the visual and perceptual information. In addition, the results of these comparisons could provide early indications of conditions that may lead the simulation -especially in the context of assistance in adult learning (i.e. learning a route or a complex spatial configuration) -an aid to reading comprehension in children.

This project is ambitious in that it can contribute to the development of new interdisciplinary knowledge about the processes implemented in the learning activities, memory and understanding.In the field of the study of reading comprehension, the theory of mental models has had a great influence in accounting for the inference process and characterizing the mental representations built. Many researchers have relied on this theory to develop new knowledge in the area of reading comprehension, but all its theoretical implications have not been fully explored and discussed. The point of view developed by Johnson-Laird (1983) is re-examined and renewed with the debates around embodied cognition. This could lead to reconsider some proposals and consider new links with cognition as a whole. The present project aims to contribute to this theoretical development. It aims to inform the nature of the mental representations developed during language comprehension and inform their relationship with the representations of the world as we can experience or imagine. Thus, this project can contribute to the development of new knowledge about the processes at work in memory and understanding, and contribute to theoretical reflections on the place occupied by mental imagery and mental simulation more generally.

This project addresses innovative questions related to the new and exciting approach of embodied cognition. This issue is highly relevant in the face of the important effort devoted to the study of language comprehension and difficulties experienced by poor comprehenders. Surprisingly, to date only a very few studies have been conducted in children despite valuable potential applications in education. The perspective of the present project is innovative in several ways: First it aims at providing new evidence of a highly debated issue, that is the role of PMS in language comprehension. Its specific contribution relies on the use of complementary methodology combining behavioral and neuroscientific methods. This project takes advantage of virtual environment technique that provides valuable methodological possibilities for testing new hypotheses. To our knowledge, this project will be the first comprehensive study on perception simulation comparing skilled and less skilled comprehenders and using virtual environments. It is a very promising issue both on the theoretical side and on a more applied one. This kind of research can indeed contribute to the debate on the importance of perceptual simulation in language comprehension and to its long-lasting effects in memory. It can in addition become a tool to help comprehenders whose difficulties could come in part from a lack of simulation efficiency.
Thus, in addition to these theoretical issues, the project will open new research directions that will allow well-needed applications of the results to various situations, including in educational and professional contexts, in particularly in populations presenting difficulties. Another related objective will consist in developing new methodological tools to assess perceptual motor simulation.

To be coming

In the last 15 years, the embodied cognition approach has become widespread in all fields related to cognition. The embodiment approach to meaning considers that language is grounded on the world which means that the same perceptual, motor and emotional brain mechanisms used in real world experience are involved to some extent in the processing of linguistic meaning (Barsalou, 1999, Zwaan & Taylor, 2006, and see for recent reviews, Coello & Bartolo, 2012, De Vega, 2012). Meaning would thus consist of the mental simulation of the objects, events and situations to which words refer to. The idea is that people understand linguistic descriptions, e.g. of an action, by mentally simulating just like people understand directly observed actions by others through mental simulation. Many studies have highlighted the involvement of sensorimotor systems in language comprehension. The perceptual and/or motor simulation (PMS) phenomena in language processing has been repeatedly confirmed through behavioral and neuroscientific studies, but the question of the role of PMS has not been solved yet. This is one of the major challenges of this project. It aims to explore the role of mental simulation in language comprehension activities by focusing particularly on the construction and memory of spatial representations in adults, and reading comprehension in children. The research program is structured in two main axis. The first axis will explore some basic properties of the PMS. It will consider situations in which neither the task, nor the instructions favor strategical mental simulation, such as simple memory tasks of lists of words or objects. The proposed experiments will question the role of motoric information carried by the concepts themselves in their memory (including the role of information on the mass of the object and affordances), in adults and children. To highlight the simulation process, the studies plan to rely on interference paradigms acting at the encoding phase or at the level of the response. It will also exploit the possibilities offered by virtual environments. A study by transcranial magnetic stimulation in adults will complement the series of experiments on the memory of affording objects. The second axis aims to study the role of mental simulation in more complex language processing activities. We will focus on some situations where simulation should be particularly encouraged and beneficial to understanding and memorization. An expert learning situation in adults will be considered with experiments on the role of PMS on the construction and the memory of a spatial model. Through the use of virtual environments and paradigms of interference, we will test the hypothesis that the PMS can facilitate and strengthen the construction and memory of the spatial model. We will also study the role of PMS in reading comprehension in children. We will ask whether simulation is related to skilled comprehension. More specifically, we will test the hypothesis that children with less skilled comprehenders may not be able to effectively implement this simulation during reading.