image-based Town On-line Web Navigation and Searchengine – ITOWNS

The goal of this project is to define and implement a new generation prototype of multimedia
web tools that mixes a broadband 3D geographic image-based browser (such as the French
Geoportail, Google Earth, Microsoft live Earth) together with an image-based search engine
with queries.
This project addresses specifically very spatially dense high resolution ground-based
imagery acquired on the street scale.
The first goal is to navigate with a browser freely and fluidly within the image flow (without 3D
models) in huge data sets to see and visit the city as if we were there.
The second goal is to build from the images a content-based information system to provide
within the browser tool some simple and more complex services and applications using
queries (goto a given address, generate hybrid text-image navigation maps, find the location
of an orphan image, select the images that can see an object, etc.).
To achieve these goals we have to address three distinct challenges.The first challenge of
this project is to visualise and navigate in an “image-based” way across the web through a
huge amount of dense georeferenced panoramic images of cities acquired by a groundbased
mobile mapping system. In the scope of this project one TerraByte of data acquired on
the city of Paris will be acquired and processed. This corresponds to 25000 georeferenced
panoramic views (composed of 10 full HD camera images) along 100 km of streets.
The second challenge is to extract from the images fully automatically and in reasonable time
as many as possible features, elementary objects, complex objects and geometrical and
topological relations between the objects for content-based retrieval.
The third challenge is to exploit and combine the objects, features and signs previously
extracted in order to provide high level semantic data comparison and efficient machine
learning solutions for scene classification and retrieval. Content-based search engine
strategy is introduced to go further on the application showcases. The development of
“smart” data mining will correspond to different levels of query complexity.
This project although very challenging in its scientific issues will also be application-driven,
as we shall propose the implementation of basic and more complex applications using the
extracted signs and the developed search engines.
The very complementary skills and expertise of the different laboratories (MATIS, LIP6,
ETIS, LCPC, CMM) in photogrammetric data collection, object extraction and pattern
recognition, machine learning and content-based retrieval, are a key to the success of this
project.