Algorithms for modeling the visual system: From natural vision to numerical applications. – KEOpS

We can summarize the scientific method in terms of an interdisciplinary platform allowing us to link (i) experimental observations with (ii) the analysis of data and models, involving fine articulation between bio-inspired distributed computing and the treatment of cerebral neural signals (encoding / decoding). Concrete results in design and implementation of natural image sequences processing were also obtained. Here are the highlights: Non-standard behavior of the dynamics of retinal cells in the presence of natural image sequences. Identification of non-linear operators modeling these non-standard sensors natural image processing. Statistical analysis of the response and neural coding in the vertebrate retina, in the framework of statistical physics. IT design and development of a digital nonstandard early vision system. Integration of these sensory modules in an experimental sensory architecture implementing the process of the pre-cortical visual system.

With three main results down the road.

Biologists of the Chilean teams mounted the experimental platform (including software tools) to study the influence of a sequence of natural images on multiple retinal ganglion celle responses. They begin to produce new experimental results that only a few international teams are able to obtain. The proof of this excellence is also given by the production of original explanatory bio-physical models.
On the French side, theoretical work has been embodied in the Enas enas.inria.fr software, making them directly usable either as an application with a graphical interface, or as middleware ford existing software. This concerns the statistical analysis of neural spikes (action potential) from several neurons, using the most general statistical models available to the scientific community today.
In addition, if the modeling proposed in this project are truly relevant, they should also apply to video sequences processing algorithms. This last step was achieved with the provision of software components (http://mnemosyne.gforge.inria.fr/lelaa) that run on real image sequences and reproduce the given functional mechanisms, including for complex visual sequences.

We consider that the main impacting results are :
-1 : showing the interest of the animal model considered in our project as a relevant model to study the Alzheimer diseases
-2 : providing a numerical demonstration of how sophisticated event detection observed as non-standard retinal could be implemented as two layer network compatible with the retinal architecture-
-3 : extending the statistical properties analysis of spike trains, beyond usual algorithmic limits of usual parametric and non-parametric methods.
- 4 : providing middle-ware for natural sequence analysis based on biologically inspired methods
- 5 : using non-trivial retinal models to enhance early-visual models of the visual brain, including in active visual robotic tasks.
With several other “secondary” outcomes : a device to record natural image sequences in realistic ecological conditions, a robotic real-time vision platform implementing visual exploration based on saccadic behavior, a software platform to experiment systemic brain model in realistic survival paradigms.

More than a dozen publications and fifty communications.

Scientific production, half of which resulting from Chilean-French common works, with multidisciplinary (neuroscience and computational) outcomes, helped to push the state of the art in both neural spike train analysis methods and on related modeling, but also on numerical experiments in order to compare the models not only to biological data but also to computer simulations. Such method allows us to be more parsimonious with respect to living matter. An experimental platform for behavioral mechanisms has been developed (http://virtualenaction.gforge.inria.fr) to also study the functional aspects of such models.
Beyond the academic production, note the production of a chapter for teaching and articles toward broader public, to share scientific culture related to these topics.

The survey of sensory biological solutions to the physics problems offers an inspiring wealthy source for the design of artificial or bio-inspired sensory systems. A better understanding of sensory coding may have fruitful applications, in the long term for people with low or impaired vision, while enactive vision is targeted in the short term. Furthermore, the whole corpus of produced knowledge is going to help better understanding how the brain works.

A Chilean and French multidisciplinary group of research teams with expertise in sensory biology; mathematical modeling, computational neuroscience and computer vision proposes to associate their complementarities to address: The integration of non-standard behaviors from retinal neural sensors, dynamically rich, sparse and robust observed in natural conditions, into neural coding models and their translation into real, highly non-linear, bio-engineering artificial solutions.

An interdisciplinary platform for translation from neuroscience into bioengineering will seek convergence from (i) experimental and (ii) analysis/models, with a fine articulation between biological inspired computation and brain neural (coding / decoding) signal processing. As a corollary, tackling modern problems in Neuroscience requires sophisticated electronic and computational equipment and provides for electronic or computer engineers and biologists in Chile and France the opportunity of new sectors of development; we thus expect as an outcome, industrial solutions, with high-level in design and implementation, performing beyond the present state of the art on degraded natural image sequences.

At a concrete level, this project is going to provide new experimental facts about non-standard retinal cell behavior in natural scenarios, the application of high-level statistical methods and tools to the design of innovative visual operators, an open-source software implementing the previous concepts at the numerical level and an experimentation platform to benchmark the obtained results.

Research topics:
-1- Dynamics of non-standard behavior of retinal cells in front of natural image sequences.
-2- Identifying non-linear mapping from natural images to non-standard sensor behavior.
-3- Statistical analysis of neural coding response in vertebrate retina, the framework of statistical physics.
-4- Computer design and numerical development of a non-standard bio-inspired early-vision front-end
- 5- Study of the integration of these new dynamic sensory modules in a visual architecture and experimental study of their performances in the case of degraded visual sources

This proposal fills the need of fostering a joint collaborative scientific and technological progress, for which France and Chile have a long tradition of cultural and scientific exchanges. However the challenge of doing science together is still pending. Until this year, all the available collaborative projects from CONICYT-CNRS-INRIA-INSERM-ECOS were for short visits and student exchanges. We see now an fantastic opportunity for doing "real" joint scientific project, and we also foresee that the area of STIC will have a tremendous impact on Education, S&T generation and transfer to a variety of areas like biomedicine, robotics and artificial intelligence.