Brain Body Digital Musical Instrument – BBDMI

Based on the musical use of EMG and EEG technologies, a team of developers, SHS researchers and music practitioners build software and hardware prototypes that they test in real artistic creation situations. The first year in the studio with experts, the second year with musicians in a learning situation and the last year in the field of experimental artistic creation presented to a general public. A company will accompany these developments to propose an open industrial design for the final prototype and all software and hardware deliverables.

Expected results.
multimodal hybrization of brain and muscle signals
Inclusive music making, via Brain-body controlled sound processing.
Prototyping design, and open development of a new digital instrument.
Open science platform for sharing project outputs with creative and scientific communities.

To share knowledge and expertise in EEG and EMG technologies in an open manner to make them accesssible for the benefit of live music creation and its various communities.

Open software libraries, block diagrams, scientific publications, artistic events.

The Brain/Body Digital Musical Instrument (BBDMI) is a Projet de recherche collaborative – Entreprises (PRCE) that will create a prototype for a digital musical instrument system that uses physiological signals from the human body: from the brain and muscles. The instrument system will be validated in a range of musical settings from concerts to the conservatoire, with a diverse range of musicians. In addition to the prototype, the project will generate soft outputs in the form of pedagogical manuals and public presentations, as well as hard deliverables such as data sets and code libraries. The project brings together an interdisciplinary team of researchers, designers, musicians and engineers that will mutually inform each other through a mixed methods approach. The consortium is led by La Maison de Sciences de l’Homme Paris Nord, with as industry partner the company, Soixante Circuits, and as partners, Paris Brain Institute (ICM), and the Centre de Recherche Informatique et Création Musicale at the Université Paris 8. The consortium represents the spectrum of research: from low-level technical development to innovative user-centered design, and the integration of state-of-the art methods integrating neuroscience and musical creation.
The work is organised into five work packages that places the user at the center of an iterative R&D process. This cycle starts with an initial scoping workshop, followed by three user studies. Group activity will generate ideas for tasks to be studied in individual trials, thereby including the user in the design of each study. Each study will test successive prototypes that inform design recommendations for the next iteration.
Technical innovations include the hybridization of brain signals (electroencephalography: EEG) and muscle signals (electromyography: EMG). Separately, EEG-based systems suffer from small signal amplitudes and low accuracy, while EMG-based interfaces suffer from subject fatigue and spasticity, especially in users that have less motor control. By combining EEG and EMG, in a new, hybrid mode of interaction, we will leverage the complementarity of brain and body signals. EMG signals can be fast, accurate, and provide an index of physical effort, while EEG signals indicate fluctuations of mental activity and (in)attention. Although it is shown that EEG-EMG hybridization enhances practical usability and accuracy, to the best of our knowledge, this has not yet been attempted in biosignal music.
Creative innovations include the investigation of mapping of hybrid EEG & EMG features to sound synthesis, sound processing and musical composition. This will result in the development of a musical toolbox of modules that can be used at different levels of experimental and creative stages of musical production. This toolbox will be developed in open-source computer music languages in a modular framework that enables perennity, portability and cross-compatibility with common music scripting environments, and allows flexible porting to microcontroller-based systems.
The project will result in impact in creative art/science, coding and makers communities through a program of public presentations and cultural mediation for which the partners are uniquely positioned. The BBDMI will result in technical and design innovations in physiological interfaces and digital audio, and will provide new personal technologies with a high degree of useability. This will catalyse a transfer of technology out of the research lab into the commercial music technology landscape. The open-source approach of the project will assure long term accessibility of core project innovations to future student, artist, and researcher communities, while leaving open the possibility of commercialization by project partners.