Ultra Low Power Smart 3D Cochlea – ULP-COCHLEA

We target the design of the first end-to-end bioinspired acoustic sensor with AI embedded system, with power consumption around 10 milli Watt .
Our goal is to test them to study marine life, from shellfish to the blue whale, and to provide data on marine life abundance, distribution, and behaviour information.
We also aim at developing ULP sensors well-tuned to detect and alert about acoustic events of interest (presence of whales and dolphins, ship passages, illegal fishing with explosives, oil exploration, etc).
The technical specifications of the bioinspired intelligent hydrophone are the following:
• Bioinspired Analog Cochlea (Bank Filter, Spikes readout through Biomimetic Neuron ): ULP (10 milli Watts), UWB (1 kHz à 100 kHz - 32 bands) with bioinspired binaural audition (gain, localisation, includes correlator)
• ULP AI chip (1 milli Watt) running an optimized Spiking Neural Network dedicated to pattern recognition capabilities of acoustic signatures
• Sensors well-tuned to detect and alert about acoustic events of interest : (1) the presence of whales and dolphins and their activity, such as social communication or hunting (2) anthropogenic noise (fishermen boats, touristic boats, illegal fishing with explosives).

Notably, this research program is based on the interdisciplinary synergy between three distant communities able to develop: ULP analog electronics (CMOS transistors operating in deep subthreshold operation), artificial intelligence (machine learning, spiking neural networks) & ULP AI chips, and bioacoustics/ecoacoustics.

This technology strongly contributes to sustainable development through accurate monitoring of the submarine ecosystem: following exposure to the noise of human activities, to pollution or ocean warming. This is made possible through the use of highly tunable ULP AI processor, which process the natural data (spike trains reflecting input acoustic signal features) output by the cochlea(s) to detect, locate, record, analyse, classify and alert acoustic events of interest (presence of cetaceans, boat passages, illegal fishing with explosives).

Ocean is the lifeblood of Earth and its biodiversity is threatened. Global warming, acidification, dead zones ... the ocean is one of the first victims of CO2 emissions, but it also protects us by absorbing it, a vital role that we must take into account in climate policies.The project outcomes will allow the design of incentive policies for environmental sustainability, to track their effectiveness over time and provide options for adjusting them.

The applicative domain is much larger than ocean ecoacoustics and concerns any kind of sounds in sea, air or ground, including bats, birds, crickets, bees, worms underground, even the ascent of sap in the trees for measuring many aspects of our biodiversity et its evolution.

Our acoustic sensor is a general-purpose cochlea that produces a «vision« of the acoustic environment. It can decode specific events of interest to produce «alerts«, possibly with some associated spatial info or record them for a long time. This is a new type of intelligent ocean passive acoustic sensors based on ultra-low power (ULP) electronics using an industrial CMOS technology (transistors operating in deep sub-threshold operation), used both for localisation circuit and binaurally (3D acoustics). Bio-inspired cochlea coupled to AI-processing, the latter implementing spiking neural network (SNN) can open new perspectives in the interpretation of the marine acoustic ecosystem.

We urgently need good affordable monitoring techniques at a very large scale to map the ocean soundscape. Such radically new and disruptive sensors would help us to monitor the marine acoustic environment, to extract information relevant for biological systems and to understand the ecosystems. We develop a new discipline, the «biology of disturbance», to guarantee to marine life and oceans a healthy future (that is beneficial to humans too). The development of innovative solutions to extend in time and space our acoustic monitoring capabilities will be beneficial for the protection of marine life, the protection of whole ocean basins, and the maintaining of marine ecosystem services that interact with the whole biosphere (e.g. CO2 fixation and segregation, O2 production).

Our small cheap and ULP sensor has a very limited environmental footprint, is totally passive, almost “invisible”, but can listen and analyse autonomously off the cloud. This smart sensor has the capability to adapt itself to the environment. It is a learning sensor which records the soundscape, classify different species or noises from their different acoustic signatures in an unsupervised manner. We can deploy them everywhere and they will last for a long time on a battery or passive energy harvesting before needing to be retrieved.

The ocean is not quite a haven of peace. The environment is noisy, the fauna is bathed in the din of icebergs, underwater seismic activity, different animal cries, waves, winds, and so on. All these sounds are the sign of a dynamic ocean, and wildlife has come to terms with it over time. Sonars affect the echolocation of cetaceans and regularly cause massive stranding. Oil exploration adds to this noise pollution when it uses “seismic”, a method that visualizes geological structures in particular in the Mediterranean Sea. This method is so intrusive that the presence of observers on board is mandatory to ensure that there are no marine mammals around the ship. Our sensor detects and alerts about acoustic events of interests (i.e. cetacean clicks, fishermen boats, etc) and observes specific species possibly with some associated spatial info and to record during long periods.

CRIStAL, Université de Lille
- P. Devienne, P. Boulet and al. VS2N: Interactive Dynamic Visualization and Analysis Tool for Spiking
Neural Networks, session Bio-inspired circuits for multimedia, CMBI, Juillet 2021
- P. Devienne, P. Boulet and al. Unsupervised visual feature learning with spike-timing-dependent plasticity:
How far are we from traditional feature learning approaches? Pattern Recognition, Elsevier, 2019, 93, pp.418-
429.
- P. Devienne, P. Boulet and al., Multi-layered Spiking Neural Network with Target Timestamp Threshold
Adaptation and STDP. International Joint Conference on Neural Networks (IJCNN), Jul 2019, Budapest,
Hungary.

IEMN, Université de Lille
- Danneville F., Carpentier K., Sourikopoulos I., Loyez C., Sub-0.3V CMOS neuromorphic technology and its
potential application, Invited Paper, Special Session “Bio-inspired circuits, systems and algorithms for
multimedia”, CBMI 2021, 28-30 June 2021
- Danneville F., Loyez C., Carpentier K., Sourikopoulos I., Mercier E., Cappy A., A sub-35 pW Axon-Hillock
artificial neuron circuit, Solid-State Electron., 153 (2019) 88-92
- Cappy A., Danneville F., Hoël V., Loyez C., Neurone Artificiel, FRANCE (Institut National de la Propriété
Industrielle), FR3050050, 13 Octobre 2017 (patent)

INPS, Université de Toulon
- Hervé Glotin and al. Inter-annual decrease in pulse rate and peak frequency of Southeast Pacific blue whale
song types, Nature, Scientific Reports, V10, 8121, 2020
- M. Poupard et al., «Real-time Passive Acoustic 3D Tracking of Deep Diving Cetacean by Small Non-uniform
Mobile Surface Antenna,« ICASSP 2019 - 2019 IEEE ICASSP, Brighton, United Kingdom, 2019.
- M. Fourniol, V. Gies, V. Barchasz, E. Kussener, H. Barthelemy, R. Vauche and H. Glotin, Low-Power WakeUp System based on Frequency Analysis for Environmental Internet of Things, IEEE MESA 2018

This project aims to develop disruptive ocean passive acoustic sensors. Fully bioinspired, ultra-low power, smart and compact, they can monitor in-situ for extended time periods while consuming less than 10 milliwatts (that is, a hundred times better than current technology).

The sensor comes from the design of an ultra-low-power (ULP) analog cochlea operating preliminary data processing (analog acoustics signals converted into spike trains), judiciously associated with ULP processors that implement spiking neural networks. These sensors are almost “invisible” and feature a low environmental footprint. They analyse data off the cloud, while being robust because fabricated in (low cost) standard CMOS technology. They can cover a large area and are maintenance-free (long battery lifetime or powered by energy harvesting).

This technology strongly contributes to sustainable development through accurate monitoring of the submarine ecosystem: following exposure to the noise of human activities, to pollution or ocean warming. This is made possible through the use of highly tunable ULP AI processor, which process the natural data (spike trains reflecting input acoustic signal features) output by the cochlea(s) to detect, locate, record, analyse, classify and alert acoustic events of interest (presence of cetaceans, boat passages, illegal fishing with explosives).

Ocean is the lifeblood of Earth and its biodiversity is threatened. Global warming, acidification, dead zones ... the ocean is one of the first victims of CO2 emissions, but it also protects us by absorbing it, a vital role that we must take into account in climate policies.The project outcomes will allow the design of incentive policies for environmental sustainability, to track their effectiveness over time and provide options for adjusting them.

The applicative domain is much larger than ocean ecoacoustics and concerns any kind of sounds in sea, air or ground, including bats, birds, crickets, bees, worms underground, even the ascent of sap in the trees for measuring many aspects of our biodiversity et its evolution.

Notably, this research program is based on the interdisciplinary synergy between three distant communities able to develop: ULP analog electronics (CMOS transistors operating in deep subthreshold operation), artificial intelligence (machine learning, spiking neural networks) & ULP AI chips, and bioacoustics/ecoacoustics.