Blanc inter SIMI 2 - Blanc international - Sciences de l'information, de la matière et de l'ingénierie : Sciences de l’information, simulation

Algorithms for modeling the visual system: From natural vision to numerical applications. – KEOpS

From natural vision to digital application: a new vision of our retina

The survey of sensory biological solutions to the physics problems offers an inspiring wealthy source for the design of artificial or bio-inspired sensory systems. A better understanding of sensory coding may have fruitful applications, in the long term for people with low or impaired vision, while enactive vision is targeted in the short term. Furthermore, the whole corpus of produced knowledge is going to help better understanding how the brain works.

Our retina has a non-standard behavior with natural stimuli

While the behavior of the ganglion cells of the retina seems well-understood during the presentation of artificial stimuli (a model with a spatial and temporal filter followed by a non-linear function is relevant presentation), when the retina is studied in the condition if natural image sequence, its behavior breaks these standards: a parsimonious and less random activity is observed, leading to sophisticated mechanisms of space-time events detection. This allows a relevant compression of information, taking into account the statistics of natural images. This corresponds to the fact there is a hundred times fewer fibers in the optic nerve than photoreceptor cells. A better understanding of this complex mechanism and new scientific challenges are addressed.<br />Specifically, the project was able to start providing new experimental data on the behavior of non-standard retinal cells in the case of natural scenarios and propose new methods and high-level statistical tools to analyze these innovative biological visual operators. Efficient computational mechanisms in computer vision have also been proposed. An open software for digital implementation of previous concepts and experimentation platform for evaluating the results is available. In terms of bio-physical models several contributions to the state of the art was provided.

We can summarize the scientific method in terms of an interdisciplinary platform allowing us to link (i) experimental observations with (ii) the analysis of data and models, involving fine articulation between bio-inspired distributed computing and the treatment of cerebral neural signals (encoding / decoding). Concrete results in design and implementation of natural image sequences processing were also obtained. Here are the highlights: Non-standard behavior of the dynamics of retinal cells in the presence of natural image sequences. Identification of non-linear operators modeling these non-standard sensors natural image processing. Statistical analysis of the response and neural coding in the vertebrate retina, in the framework of statistical physics. IT design and development of a digital nonstandard early vision system. Integration of these sensory modules in an experimental sensory architecture implementing the process of the pre-cortical visual system.

With three main results down the road.

Biologists of the Chilean teams mounted the experimental platform (including software tools) to study the influence of a sequence of natural images on multiple retinal ganglion celle responses. They begin to produce new experimental results that only a few international teams are able to obtain. The proof of this excellence is also given by the production of original explanatory bio-physical models.
On the French side, theoretical work has been embodied in the Enas software, making them directly usable either as an application with a graphical interface, or as middleware ford existing software. This concerns the statistical analysis of neural spikes (action potential) from several neurons, using the most general statistical models available to the scientific community today.
In addition, if the modeling proposed in this project are truly relevant, they should also apply to video sequences processing algorithms. This last step was achieved with the provision of software components ( that run on real image sequences and reproduce the given functional mechanisms, including for complex visual sequences.

We consider that the main impacting results are :
-1 : showing the interest of the animal model considered in our project as a relevant model to study the Alzheimer diseases
-2 : providing a numerical demonstration of how sophisticated event detection observed as non-standard retinal could be implemented as two layer network compatible with the retinal architecture-
-3 : extending the statistical properties analysis of spike trains, beyond usual algorithmic limits of usual parametric and non-parametric methods.
- 4 : providing middle-ware for natural sequence analysis based on biologically inspired methods
- 5 : using non-trivial retinal models to enhance early-visual models of the visual brain, including in active visual robotic tasks.
With several other “secondary” outcomes : a device to record natural image sequences in realistic ecological conditions, a robotic real-time vision platform implementing visual exploration based on saccadic behavior, a software platform to experiment systemic brain model in realistic survival paradigms.

More than a dozen publications and fifty communications.

Scientific production, half of which resulting from Chilean-French common works, with multidisciplinary (neuroscience and computational) outcomes, helped to push the state of the art in both neural spike train analysis methods and on related modeling, but also on numerical experiments in order to compare the models not only to biological data but also to computer simulations. Such method allows us to be more parsimonious with respect to living matter. An experimental platform for behavioral mechanisms has been developed ( to also study the functional aspects of such models.
Beyond the academic production, note the production of a chapter for teaching and articles toward broader public, to share scientific culture related to these topics.

The survey of sensory biological solutions to the physics problems offers an inspiring wealthy source for the design of artificial or bio-inspired sensory systems. A better understanding of sensory coding may have fruitful applications, in the long term for people with low or impaired vision, while enactive vision is targeted in the short term. Furthermore, the whole corpus of produced knowledge is going to help better understanding how the brain works.

A Chilean and French multidisciplinary group of research teams with expertise in sensory biology; mathematical modeling, computational neuroscience and computer vision proposes to associate their complementarities to address: The integration of non-standard behaviors from retinal neural sensors, dynamically rich, sparse and robust observed in natural conditions, into neural coding models and their translation into real, highly non-linear, bio-engineering artificial solutions.

An interdisciplinary platform for translation from neuroscience into bioengineering will seek convergence from (i) experimental and (ii) analysis/models, with a fine articulation between biological inspired computation and brain neural (coding / decoding) signal processing. As a corollary, tackling modern problems in Neuroscience requires sophisticated electronic and computational equipment and provides for electronic or computer engineers and biologists in Chile and France the opportunity of new sectors of development; we thus expect as an outcome, industrial solutions, with high-level in design and implementation, performing beyond the present state of the art on degraded natural image sequences.

At a concrete level, this project is going to provide new experimental facts about non-standard retinal cell behavior in natural scenarios, the application of high-level statistical methods and tools to the design of innovative visual operators, an open-source software implementing the previous concepts at the numerical level and an experimentation platform to benchmark the obtained results.

Research topics:
-1- Dynamics of non-standard behavior of retinal cells in front of natural image sequences.
-2- Identifying non-linear mapping from natural images to non-standard sensor behavior.
-3- Statistical analysis of neural coding response in vertebrate retina, the framework of statistical physics.
-4- Computer design and numerical development of a non-standard bio-inspired early-vision front-end
- 5- Study of the integration of these new dynamic sensory modules in a visual architecture and experimental study of their performances in the case of degraded visual sources

This proposal fills the need of fostering a joint collaborative scientific and technological progress, for which France and Chile have a long tradition of cultural and scientific exchanges. However the challenge of doing science together is still pending. Until this year, all the available collaborative projects from CONICYT-CNRS-INRIA-INSERM-ECOS were for short visits and student exchanges. We see now an fantastic opportunity for doing "real" joint scientific project, and we also foresee that the area of STIC will have a tremendous impact on Education, S&T generation and transfer to a variety of areas like biomedicine, robotics and artificial intelligence.

Project coordination

Thierry VIÉVILLE (INRIA Centre Nancy Grand-Est) –

The author of this summary is the project coordinator, who is responsible for the content of this summary. The ANR declines any responsibility as for its contents.


INRIA Sophia Antipolis - Méditerranée INRIA Centre Sophia-Antipolis
INRIA - NGE INRIA Centre Nancy Grand-Est

Help of the ANR 296,053 euros
Beginning and duration of the scientific project: - 36 Months

Useful links

Explorez notre base de projets financés



ANR makes available its datasets on funded projects, click here to find more.

Sign up for the latest news:
Subscribe to our newsletter