Knowledgeable and Multimodal Geographic Large Language Models Grounded with Reasoning and Retrieval – Geo-R2LLM

Recent Artificial Intelligence (AI) research has given rise to a paradigm shift brought by Large Language Models (LLMs). Though LLMs arose from research in Natural Language Processing (NLP), it is well-known today that zero-shot and few-shot transfer learning methodologies as well as novel prompting strategies make their deployment possible beyond the NLP field, achieving impressive performance on a significant range of domains and downstream tasks. However, the deployment of LLMs in geographic information systems is still in its infancy.
The Geo-R2LLM project aims to create a novel paradigm for building knowledgeable and multimodal geographic LLMs by rethinking LLMs generation mode with retrieval and reasoning over multiple multimodal external knowledge sources to ground predictions. The improved multimodal geographic LLMs will be integrated in a geospatio-temporal AI (GeoAI) system prototype and evaluated on a pilot application related to context-aware navigation systems in a complex urban environment. Navigation services can be considered as one of the most critical and widely adopted location-based services in modern society, hence the project has potentially strong impact also outside of academia.
This research will lead to fundamental advances in multiple disciplines spanning GeoAI, spatio-temporal reasoning, information retrieval, and natural language understanding, laying the groundwork for more effective AI platforms for various domains that relate to geography and geographical information science.