Prosody: Accents, Dialects, Expressivity – PADE

In order to describe and characterise the prosodic variations, we aim at evaluating a set of distances measures for prosody, in order to compare utterances. Prosodic variability is understood as a functional approach of this framework, including linguistic, paralinguistic and non-linguistic information. Moreover the high variability linked to prosodic transfers between native and second language, or between standard and vernacular language, or either the intercultural variability linked to social affects have to be considered. Therefore, such distances measures will have to account for very diverse situations and constraints. However as prosody is one single object, and received as a single object by listeners, encoded in one given acoustic material, a holistic approach has been preferred.
Perception tests aiming at assessing these distances will be a fundamental aspect of the project. They will provide a solid ground for evaluating the pertinence and efficiency of the proposed measurements. It will also be of prime importance to confront the results obtained using the proposed distances to those obtained using descriptors found in the literature. Therefore, prosodic models, already precisely tuned according to the main prosodic functions, will be used as a reference for the relevancy of our measures. Dialectological atlases could also be used as a reference of the expected inter-dialect distance.
It will be necessary to improve analysis methods for the physical parametes of prosody so as to yield reliable measured data. Prior to any stylisation or modelling of prosodic variation, this preliminary step still suffers serious shortcomings. Except the extraction of /F/0 and intensity, for which the existing procedures are trustworthy, prosodic parameters all raise specific issues. The most complex one seems to be the measure of voice quality, for which work in our laboratory should allow for tracking a number of parameters to be taken into account in prosodic distances and to be assessed throughout synthesis and comparative perceptual evaluation procedures. Another challenge will be to integrate multimodal parameters into prosodic distances: the variability of the dynamics and measurement scales for these parameters, as well as their relative perceptual and their timing should be taken into consideration to obtain convincing results. The use of machine learning methods would enable the contribution of each of these characteristics to be weighted, this task to be facilitated and this holistic approach to be given all its reason for being.
The major scientific stakes of this project concern the establishment of calibrated measures. Especially for prosodic modelling, this project will allow to improve the definition of prosodic dialectic atlases; to better describe the diachronic evolution of journalistic style; to be able to describe social affects using prosody and multimodal information in a temporal manner. Beyond the repercussions of these measures, we propose to improve the description of acoustic features for voice quality, which constitutes one of the main obstacles to affects understanding. Moreover, we will have to describe the links between the audio and video modalities in the framework of prosodic expressivity. A better comprehension of intermodal co-occurrences as well as links between prosodic variations and linguistic structures of utterances, achieved thanks to the measures, should help to understand the encoding differences between attitudes and emotions.
The realisation of this project mean to answer a set of fundamental questions. The first question is about prosodic distances : we will have to formalise, on the basis of known methods as dynamic programming, how to take into account the temporal information using different time basis, in both the audio and video modalities.