Artificial Intelligence foR Semantically controlled SPEech UndeRstanding – AISSPER

Artificial Intelligence (AI) is of national strategic importance due to the impressive results of deep learning algorithms in different domains such as natural language processing (NLP), communications, medicine, law, political analytics, and military with a wide range of applications. France is becoming an important leader in the field thanks to recent political efforts as pointed out is recent reviews. During the last decade, large efforts have been devoted to end-to-end spoken language understanding (SLU) systems motivated by the feasibility of popular applications such as personal assistants and conversational systems. Superior results have been observed with these systems in automatic speech recognition (ASR) with architectures based on a complex number algebra, called quaternions, requiring fewer processing time (Morchid 2018) and parameters to be estimated compared to models based just on real numbers (Parcollet et al. 2018; 2019). Reduction of model parameters makes it possible to effectively train neural architectures with limited amounts of data, often difficult to obtain for concepts and conversation semantic contexts in specific unconventional domains. Problems of inherently sequential nature, such as ASR and SLU, preclude parallelization within training examples, which becomes critical at longer sequence lengths, as memory constraints limit batching across examples. Furthermore, error analysis, conducted on the results of completed projects such as M2CR, JOKER, VERA, SUMACC, Media or DECODA, has shown the importance of leveraging on prior domain knowledge for semantic interpretation. This project will investigate novel attention models that use semantics to focus on specific contextual information for improving concept classification.
Context of AISSPER: Effective modeling of variabilities, exhibited by human’s speech at phoneme, word, and sentence levels, is still an open research problem for SLU systems. Even more, extracting relevant keywords, topics or concept mentions from either a sentence or an entire spoken document is a difficult task even for the most advanced end-to-end systems. Moreover, for speech signals, the recording conditions and the paucity of domain-specific data make it difficult to extract relevant information in different contexts without the use of external knowledge, such as domain-specific ontologies. Another crucial problem for the available solutions is the interpretability and robustness of neural based SLU systems. The need for addressing these problems in systems for SLU have been recently addressed in the IRASL workshop at NIPS 2018. It is thus proposed to make explicit selection and evidence of appropriate relevant contexts and the relative uncertainty for enhancing interpretability and improving robustness.
AISSPER aims to develop new paradigms that jointly model sentence-level and global conversation-level semantic features to understand spoken documents. Specifically, AISSPER will develop new neural attention mechanisms for improved end-to-end neural SLU systems at the sentence level and at the global document level. To achieve that, AISSPER forges a strong collaboration between established researchers from multiple disciplines: speech and language processing, and machine learning from LIUM, LIA (Academics) and Orkis (Industrial). Furthermore, the proposed research will continue to benefit from the collaboration between LIUM and the MILA Institute actually focusing on attention based machine translation in the M2CR European/Canadian project, and between the LIA and the MILA Institute for the development of complex-valued neural networks.