Structured, Consolidated and Unified Interactive Dialogue for Robot Task Performance – DISCUTER

Robots that can learn new concepts and be instructed to carry out novel tasks simply via conversation---as opposed to full demonstration or hard-coding by experts---will be valuable in a wide array of contexts in the future, from performing construction tasks on factory floors to engaging in physical conflict situations. Such robots will need to be able to react on the spot to new information that they acquire either through observation of the external context or through conversation. Developing truly conversational assistants of this sort will require modeling how the meaning of what is said in a conversation, and what is going on in the non-linguistic context in which it is situated, dynamically interacts with a robot’s knowledge base and representation of its own and its interlocutor’s beliefs and goals.

The main contribution of DISCUTER (Dialogue Interactif Structuré, Consolidé et Unifié pour la réalisation de Tâches En Robotique/Structured, Consolidated and Unified Interactive Dialogue for Robot Task Performance) is to study and advance the models and processes needed to effectively integrate natural language dialogue with the decision-making processes involved in performing joint human-robot and, more generally, human-system tasks. The long-term goal is to achieve the ability to use natural language dialogue to collaborate effectively with humans and to learn by interacting with them. The immediate goal is to produce a dialogue module capable of managing a conversation using extra-linguistic context elements and representations of (dynamically changing) belief states.

Current work in robotics that exploits cognitive models to guide a robot’s actions leave little to no room for the role that language can play in updating an agent's representation of its own mental state or that of its interlocutor. Vocal assistants and chatbots currently on the market can engage in more or less complex conversations, but they are rigid and fragile. They use either limited and static knowledge bases or language models that are determined by the probability of next moves without any concept of the constraints those moves put on the conversation and future moves, or of cognitive states and discourse goals of conversational participants. Much less can they capture the relation between such states and the conversation at hand.

Key to developing more flexible, reactive assistants, be they embodied as robots or not, will be to give them the tools to recognize the difference between a dialogue act or nonlinguistic event that is unexpected or unknown on the one hand and an event that is incoherent with its knowledge base and representation of its own and its interlocutor’s mental states on the other. For this, the members of DISCUTER, including specialists in language (LINAGORA and IRIT) and human-robot interaction (LAAS), will exploit work on discourse structure and interpretation, which will give us the tools to model the coherence between discourse and nonlinguistic events and the contents of knowledge bases and cognitive states. DISCUTER will use these tools to develop a module for interactive, situated conversation that will apply to any multimodal system, whether it is a fully embodied robotic assistant or a voice assistant. The potential application areas for such a module are numerous, and exist wherever natural language communication would enhance intelligent automation.