Acquisition, Treatment and Rendering Of Complex Objects – ATROCO

Algorithms for realistic image synthesis have numerous applications, for example in cultural heritage,
special effects for movies, virtual prototyping and video games. A common theme for all these
applications is that they require more and more realistic images, with more realistic models. But these
realistic models carry a high complexity, as they are usually defined by a polygonal mesh with a very
large number of polygons. A consequence of this complexity is that they are hard to represent: it takes a
professional modeller several days or weeks to create a 3D model of an existing complex object such as
a statue. In addition to the geometric model complexity, the reflectance field of the object also needs to
be precisely represented. Generally, textures are used to this end. However, realistic textures turn out to
be extremely complex, as they define a six-dimensional function, representing a bi-direction reflectance
property on each individual surface point. Together, geometry and reflectance raise significant problems
in terms of storage and real time processing.
Our key idea in this research project is to revolutionize the classical modelling pipeline for realistic image
synthesis: instead of having a person create a polygonal model of an existing object, we want to create
the model directly from the object. Although several tools exist for creating digital models of existing
objects, they currently require constant input from expert users, carefully controlled lighting and a long
post-processing step. The models created are high-definition polygonal models, unsuitable for real-time
display on standard graphics cards, and the appearance of the object, its colours, materials and textures
must be captured separately by another process. Our goal is therefore to simplify the whole process of
model acquisition and rendering:
- Develop an acquisition method that requires little user input, and can therefore be used by
novice users. This acquisition method must capture both the geometry and the appearance of
objects.
- Post-process the models created by the first step to make them suitable for real-time display,
using a combination of geometric and image-based representation.
- Render these models and integrate them into virtual worlds. This requires illumination simulation
on these complex models. The sheer complexity of the models makes rendering them a very
difficult task. Simulating the propagation of light inside these models and their interaction with a
virtual environment is even more difficult.
These three tasks are more inter-dependent than they appear at first sight: for example, the acquisition
of the geometry of the models makes assumptions on the material of the model and on the lighting.
Being able to conduct an illumination simulation on the model will validate these assumptions and help in
creating a more accurate model.
Our research project has several applications. The main application we envision is with cultural heritage:
both for digital archival of items and for displaying these items with the public. But our project also has
potential uses with all the applications that require the presence of a real object inside a virtual world,
such as augmented reality, special effects and video