VLA + MLMLMLM

The Muscle-Listening Machine Learning Model for Live Music (MLMLMLM) is a novel deep generative model for EMG-conditioned audio sequence generation that predicts viola-like sound based solely on the bioelectric signals of my muscles as I play viola.

In essence, the performance is a superimposed soundworld of what I actually play and what I probably play, based solely on the bioelectricity running through my muscles, a study of musical meaning within and outside of the performing body

dataset

MLMLMLM is trained on a custom dataset of time-aligned audio and electromyographic signals that I collected from my own body as I played viola during a residency at the The Studio for Electroacoustic Music, Akademie der Künste in Berlin, July 2025. I wrote about the residency on Substack.

With a human-centred approach to computing, VLA + MLMLMLM brings to light experiential dimensions of viola playing that are typically hidden from the audience. As such, we listen through emergent aesthetics of machine listening -- through muscles.

publication:

Strauss, L., Thattai Ravikumar, P., Yee-King, M. ‘Cross-Modal Sig2Sig Machine Translation with Deep Generative Modeling for NIME Design’. International Conference on New Interfaces for Musical Expression, 2026. [https://doi.org/10.5281/zenodo.20784411]

︎ GitHub repository

︎ newsletter / blog