Looking for relevant linguistic units to improve the intelligibility measurement of speech production disorders – RUGBI

For this, RUGBI relies on important corpora, and already available speech productions of healthy subjects (190) and patients (365) presenting pathologies of structural origin (VADS cancers) and neurological (Parkinson's disease), in the execution of different linguistic tasks, and for a part of them, at different stages of the disease. These corpora are a considerable asset for the conduct of the two areas of study of the project, respectively based on i) the perception of the speech intelligibility and ii) the automatic speech processing modeling, and more particularly, on the Deep Learning and its data representation properties that will have to be exploited here. In this context, the central objective of the project brings together the expertise of its members from the medical field, the field of language sciences and speech and language engineering to meet the challenges of biology and health. The multidisciplinary expertise thus acquired by RUGBI is a guarantee of success.

The expected result is to uncover the relevant acoustic and prosodic characteristics/units from the speech signal and to identify their relative contribution to speech intelligibility. At the end of the project, models of extraction of such units and their associated measures will be produced. The therapeutic care will thus be able to focus on these relevant acoustic/phonetic and prosodic units, and allows the patient an optimal and prompt improvement, which avoids the pitfall of discouragement usually experienced during active speech remediation. The measurement of intelligibility taking into account both acoustic/phonetic and prosodic skills will be an undeniable added value, considering the importance of prosody in communication and the lack of tools currently available in clinical assessment and rehabilitation.
In parallel to computer developments provided at the end of the doctorates, the project will provide, in addition to the subjective evaluation, a number of practical tools that can be used directly by physicians and logopedists in the form of lists, sentences and calibrated results. That could improve the care process of patients by means of a rapid and objective evaluation carried out by an automatic process, permitting repeated, reliable and reproducible measurements in pre- or post-therapeutic treatments, during rehabilitation sessions or for the follow-up of the progression of the disease.
The results and of this project will be valued and promoted on the OSIRIM platform in the context of the PAROLOTHEQUE consortium. PAROLOTHEQUE is a Scientific Interest Group that aims at allowing the access of clinical corpora to scientific projects. This platform will permit the storage of all patients’ speech recordings and suitable enriched information, including for instance the intelligibility metrics developed in the RUGBI project or any other kinds of relevant information (the relevant acoustic and prosodic units).

Once validated in clinical practice, the expected results will permit the development of tools that can be handled on smartphone or tablets connected to the clinical database in order to propose a personal training for patients with a set of progressive exercises and some feedback to physicians between each appointment.
The results of the RUGBI project concerning intelligibility assessment in speech production disorders could be a basis to establish a national consensus shared and adopted by a large part of the logopedists and specialists in speech disorders. In order to reach such a goal, it will be necessary to enlarge the numbers of teams involved in active centers in France but also in Belgium, Switzerland, Québec, French speaking Africa, etc.
The expected results could also have a great benefit in the field of automatic speech processing. Indeed, performance of the automatic speech recognition (ASR) systems is dramatically degraded with moderate and severe patients with speech disorders, due to the lack of intelligibility. However, ASR systems are still seen as attractive Alternative and Augmentative Communication tools when other motor disabilities are present in addition to speech disorders (for home control for instance). Novel and original knowledge about speech intelligibility could permit the enhancement of ASR system robustness, typically by focusing attention on the targeted acoustic units, as brought by the RUGBI project (prosodic units are not involved in current ASR systems).
Regarding clinical perspectives, the possibility to use automatic tools for assessing speech disorders will allow to revisit the classification of speech disorders and to improve the aetiological diagnostic performance by acoustic approaches.

The RUGBI project and results will be presented at the European level in the European Laryngological Society Congress, in the Union of European Phoniatrician Congress (UEP) and in the French Phoniatrics Society, with a publication in the Revue of laryngologie otologie rhinologie (Bordeaux). The project and research studies will be presented at a national level in next “Journées de Phonétique Clinique” workshop and “Journée d'Études sur la Parole” conference.
The corpora used in the RUGBI project will be presented in the international LREC conference in the 2020 edition. The corpus will be managed by GIS PAROLOTHEQUE: this structure will aim at facilitating the access of pathological speech for researchers and at promoting the available data.
The research on pathological intelligibility will lead to publications in international conferences (such as 19th International Congress of Phonetic Sciences in 2019, INTERSPEECH each year) but also on speech and automatic processing such as ELSEVIER Speech Communication, IEEE Transactions on Audio, Speech, and Language Processing, Journal of Speech, Language, and Hearing Research.
The first clinical outcomes of this project will be presented at the 31th World Congress of the international Association of Logopedists and Phoniatrics (IALP) in Taipei Taiwan, august 2019. A special session will be organized by the Educational committee for Phoniatrics on the intelligibility of speech facilitating publications in the journal of this association: “Folia Phoniatrica”.

In the context of the speech production disorders observed in ENT cancers, neurological, sensory or structural pathologies, the objective of the RUGBI project is to make improvements to the measurement of the intelligibility deficit. Indeed, speech production disorders can lead to a serious loss of intelligibility, making it difficult for patients to communicate with those around them and limiting their professional and / or social life. Classically, the clinical evaluation of intelligibility is based on a global perceptual evaluation deemed unsatisfactory by its subjectivity, lack of precision and duration that lead to erroneous measures of patient’s intelligibility. In addition, speech production tasks dedicated to this type of evaluation (repetition of words, sentences, reading) are far from being adapted to a precise measure of intelligibility and allow only an overall evaluation of the speech functional impairment.
RUGBI proposes to overcome these limitations by developing a new objective evaluation tool based on i) the identification of relevant linguistic units from an acoustic and prosodic point of view, and ii) the identification of sensitive linguistic tasks. The objective of the RUGBI project is thus to supplement the therapist's tools with a precise, robust and rapid measurement allowing to develop an optimized therapeutic project with a view to a tangible improvement of intelligibility.
For this, RUGBI relies on important corpora, and already available speech productions of healthy subjects (190) and patients (365) presenting pathologies of structural origin (VADS cancers) and neurological (Parkinson's disease), in the execution of different linguistic tasks, and for a part of them, at different stages of the disease. These corpora are a considerable asset for the conduct of the two areas of study of the project, respectively based on i) the perception of the speech intelligibility and ii) the automatic speech processing modeling, and more particularly, on the Deep Learning and its data representation properties that will have to be exploited here. In this context, the central objective of the project brings together the expertise of its members from the medical field, the field of language sciences and speech and language engineering to meet the challenges of biology and health. The multidisciplinary expertise thus acquired by RUGBI is a guarantee of success.