Toward multidimensional analysis and modeling of the form-function relationship in conversational feedback
The general objective of this project is to propose a fine grained model of the form/function relationship concerning feedback phenomena in conversation. To succeed in this general objective, we need to achieve: (i) a fine-grained analysis of the different dimensions involved, prosody, lexical markers, acoustic non-verbal signals, facial expressions, head movements and co-verbal gestures, (ii) a fine-grained analysis of the communicative functions related to feedback, a rich characterization of two crucial contextual parameters: discourse context, and production context and (iv) a detailed integration of these ingredients into a general model. We consider that the truly multi-dimensional nature of the analysis proposed is an important and ambitious step for linguistic studies.
The scientific core of the project is composed of four main phases: creating some complementary corpus and prepare the data for the analyses (Task 2); Work out the linguistic analyses and create the related annotation protocols and guidelines (Task 3); Manage the annotation and analysis process and extract automatically relevant linguistic features; (Task 4-5) ; Design a model from qualitative and quantitative analysis of the annotations produced (Task 6) ; Evaluate the models and their ability to be generalizes (Task 7). More management related tasks are the Coordination and Dissemination Task (Task 1).
An important expected result is description and a multi-dimensional analysis and model of feedback in French conversations. This model will be reliable enough to be used as an input for more applicative systems such as human-computer interaction (spoken dialogues interfaces, talking-heads, avatars,...) and processing of multi-modal data (video indexing, information extraction , meetings browsers,...).
Another product of the project will be the resources that have either created or enriched in the course of the project. We are committed to make them available to the community through our Resource Center for Describing Spoken Language (SLDR). Moreover the data and the annotation will be available in the emerging standard format for language resources (ISO TC37/4).
We consider that the truly multi-dimensional nature of the analysis proposed is an important and ambitious step for linguistic studies. The experience aquired and the methodologies developed open avenues for more linguistic studies along these lines on other phenomena than feedback.
Moreover, the consideration of different situations of communication in this kind of precise study is also new and will allow to account for communicative situation variability from a more theoretical and experimental approach that what is done usually. This aspect should also be developed in other studies.
Finally, this project aims at validating and tuning a methodology and protocols that will be applicable for other kind of populations and languages in the future. We aim in particular at collaboration with people working on language pathologies and with international collaborations to allow rich cross-linguistic studies.
Prévot, L., & Bertrand, R. (2012). CoFee-Toward a multidimensional analysis of conversational feedback, the case of French language. In Feedback Behaviors in Dialog. Interspeech Satellite Event, Portland, September 2012.
In a conversation, feedback is mostly performed through short utterances produced by another participant than the main current speaker. These utterances are among the most frequent in conversational data. They are also considered as crucial communicative tools for achieving coordination in dialogue. They have been the topic of various descriptive studies and often given a central role in applications such as dialogue systems. The present project addresses this issue from a linguistic viewpoint and combines fine-grained corpus analyses of semi-controlled data with formal and statistical modeling. At the formal level, the dynamic turn of semantics has laid the ground for rich formal approaches of discourse in which the semantic/pragmatic interface can play its role. Such models allow for a rich yet precise characterization of dialogue communicative functions. The impoverished aspect of the linguistic material in these utterances allows for a truly multi-dimensional analysis that can unveil how different linguistic domains (morpho-syntax, prosody, visual channel) combine to convey meaning and achieve communicative goals. Statistical model will go in hand with the formal model giving it more empirical support and allowing it to focus on truly representative and reliable phenomena. The statistical model will be adapted to create a classification system determining the main communicative functions of a feedback item based on its properties on the different observable dimensions and contextual information. Human-systems interfaces as well systems processing audio and video data have reached interesting achievements in the domain of feedback but is waiting for more linguistic models to continue to improve. A better understanding of the form/function relation of feedback can help extracting a wide range of elements from conversational data such as decisions taken, information accepted, opinion changes or humor-involving sequences.
Monsieur Laurent PREVOT (Laboratoire Parole et Langage) – firstname.lastname@example.org
The author of this summary is the project coordinator, who is responsible for the content of this summary. The ANR declines any responsibility as for its contents.
LPL Laboratoire Parole et Langage
Help of the ANR 149,448 euros
Beginning and duration of the scientific project: September 2012 - 36 Months