Sound metaphors – MetaSon

How can we help improving the driver’s control of silent vehicles? How can we alert pedestrians to potential dangers caused by silent vehicles? My pen knows the graphical quality of my gesture; how can it give me the related feedback? These questions, which at first glance might seem dissociated, arise from a general problem that aims at defining the best strategy to inform people on the temporal evolutions of a dynamic system (here the vehicle or the writing gesture) within a specific cognitive context (in the situation of pedestrians, conductors or writers). The sound represents an incontestable sensory modality to address such questions. In fact, sounds are intrinsically linked to time and therefore perfectly adapted to the expression of dynamic characteristics. Moreover, sounds bring additional information about things that are outside our field of view in a natural way, hence revealing the invisible.

The “MetaSound” project proposes a genuine investigation of fundamental questions for an optimal use of sounds in the context of augmented reality by the means of two problems that are both rich and challenging because of the imposed constraints and the industrial and societal consequences that they induce:
- Which sounds for tomorrow’s cars? The purpose of this task is to prepare for the acoustic rupture linked to novel traction chains (hybrid/electrical) and to propose sounds adapted to the expectancies, the security and environmental constraints to improve the control of the vehicle (interior car sounds) and to protect the vulnerable persons (exterior sounds).
- Which sounds for a sensory feedback linked to writing gestures? Dysgraphic disorders are most often linked to a lack of coherency between the writing gesture and the graphical task. The sound, because of its dynamic inherency and its lucidity, is an appealing modality to sonify the gesture and hereby help children overcome their handicap by improving their dynamic trajectories when writing.

The scientific milestones of the MetaSound project are based on three fundamental questions:
- What should be sonified? (the input variables of the system and their associated rules)
- Why sonify? (the cognitive context and the sensory expectancies)
- How to sonify? (the genesis of sound metaphors)

To exhaustively treat these questions, a strong multi-disciplinary interaction is needed. The consortium gathered around the MetaSound project unites the necessary competences, hereby enabling a formal and original approach to the problem by taking into account human factors and by using a genuine language of sounds constructed thanks to the ductility of digital synthesis and the semiotics of sounds.

The project is divided in six strongly interacting tasks, which susceptibly will lead to theoretical, technical and applied advancements.
In addition to the coordination task, two main tasks concern the applications of the project and will be validated by the realization of a sonified, rolling prototype (car application) and a prototype to assist dysgraphic persons.
Two more fundamental tasks will establish original models and methods to characterize the dynamic contents of the sound signals and their reproduction by synthesis. They will allow the construction of sound metaphors and the conception of high-level control processes that act on attributes linked to sensations.
The last task is strongly technological, since it concerns the realization of a real-time synthesis platform of spatialized sounds controlled by external variables and specific cognitive scenarios to the intended applications. It will constitute a very general tool disposed for dissemination in any community linked to augmented reality and applications to interactive sounds (cinema, video games, …).