DS0804 -

Modeling learning and expertise through a bio-inspired neural network – BeatingRogerFederer

Beating Roger Federer

Modeling visual learning and expertise through a bio-inspired neural network

General objective

The goal of this project is to produce a model of human perceptual learning and expertise, that would challenge and even defeat an expert. We will use a situation in which a ball’s trajectory has to be anticipated. Using a bio-inspired approach mimicking the human visual system properties, the network should be able in a first step to determine if the ball has landed in or out the field. Then, it should be able to predict, upon a short presentation of the initial part of the trajectory, if the ball will land in or out. <br />The performances of the network will be compared to human performances, of novices and experts in tennis (players and referees), with the goal to model the visual expertise acquisition of human observers, and therefore let us know more about the processes of learning and expertise development of human beings.

To carry out this project, we record bullet trajectories using a camera that reproduces the functioning of the human retina, perceiving differences in contrast between two close instants. This information, these spikes, are then sent into a network of bio-inspired neurons. Unlike deep learning, the neurons do not receive a learning rule, with the exception of a biological rule called Spike Timing Dependent Plasticity (STDP). This rule leads to a modification of synaptic weights, which has the particularity of being unsupervised and dependent on the exact temporality of the spikes (Post and Presynaptic). This rule applied to spike neural networks (SNNs) allows neurons to become selective to repetitive patterns that have characteristics close to those observed in humans or animals.
The performance of the neural network is then evaluated when presented with variable parts of the trajectory, and compared to that of humans, novices or experts.

Simple trajectory acquisitions allowed us to have a neural network estimating with a precision of only a few pixels ball trajectories. We are currently in the process of acquiring more complex trajectories and comparing them with human performance.

We still have to make the trajectories more complex, and synchronize several cameras to allow binocular vision.

In progress

How do we turn into an expert in a perceptual task, for example, how can Roger Federer predict so fast and accurately if a tennis ball will bounce in or out the field? The goal of this project is to build a model of perceptual learning by humans, that will challenge – and even defeat – the visual capacities of an expert. This model will be bio-inspired, implying that it will function by mimicking the properties of the human visual system. In particular, three of its properties will be implemented in the network:

(1) We will use cameras that imitate the functioning of retina, and are for this reason called bio-inspired cameras or artificial retinas Such a camera sends its message, called a spike, only when detecting a change between two successive frames
(2) In a given neuronal layer, the neuro that will receive these spikes first will get its activity increased (in terms of synaptic weights), a process called Spike-Timing-Dependent-Plasticity (STDP), analog to the process of Long Term Potentiation by Humans
(3) Finally, the neuron that will endure the Long Term Potentiation will inhibit the surrounding neurons of the same layer, process called Long Term Depression. This will ensure that all neurons would not answer at the same visual property of the stimulus, and creating therefore a neuronal selectivity of some neurons at a given visual property.

Once the model will be built, by presenting numerous tennis ball trajectories, we will determine the ability of the model to predict the landing point of the ball. These performances will be compared to human performances, novices and experts in tennis. Five experimental objectives will be assessed:
(1) Evaluating the network accuracy compared to Humans, after a full or partial presentation of the ball trajectory, for example after a presentation of only 10% or 50% of the trajectory
(2) Evaluating if the performance differs if the ball comes from a ball launcher or an opponent strike. This manipulation will be done in order to determine if the network can make use of the opponent kinematic to predict the landing point, as tennis experts do, or only use information related to the ball trajectory itself
(3) Evaluate the « road to excellence », that is, or is built the expertise and performance increases after the presentation of a few hundreds of trials vs thousands
(4) Assess if performances increases if the network is fed with binocular or even tri-ocular monocular information on ball trajectory, rather than a monocular point of view. Again, human participants will be compared to these results for the monocular and binocular conditions)
(5) Determine during the learning period the exact influence of feedback that will be given to the network, and to which extent its effectiveness can be increases by varying the timing of occurrence and its frequency

This project will allow to gain a better understanding on perceptual learning and the development of expertise by Humans, enabling better fundamental and applied knowledge on this topic. In addition, it should also demonstrate the contribution and effectiveness of bio-inspired developments, with real time functioning and in-vehicles systems properties that could be transferred to new domains of application.

Project coordination

Robin Baurès (Centre National de la Recherche Scientifique / Laboratoire Cerveau et Cognition)

The author of this summary is the project coordinator, who is responsible for the content of this summary. The ANR declines any responsibility as for its contents.


CNRS/CerCo Centre National de la Recherche Scientifique / Laboratoire Cerveau et Cognition

Help of the ANR 180,999 euros
Beginning and duration of the scientific project: March 2017 - 48 Months

Useful links

Explorez notre base de projets financés



ANR makes available its datasets on funded projects, click here to find more.

Sign up for the latest news:
Subscribe to our newsletter