Real-Time Piano Onset and Velocity Detection

I’m happy to present my last work, accepted at the EUSIPCO 2023 Conference:

ONSETS&VELOCITIES builds on some good ideas from previous literature to develop a convolutional neural network with several (IMHO nice) properties that allow to perform real-time transcription of piano onsets and velocities on modest hardware (e.g. a regular laptop), while keeping very good performance!

Below is a qualitative example. I invite you to read the paper for more technical details (and cite it if it is of any use to you 😎 ):

@inproceedings{onsvel,
      title={{Onsets and Velocities}: Affordable Real-Time Piano Transcription Using Convolutional Neural Networks},
      author={Andres Fernandez},
      year={2023},
      booktitle={{EUSIPCO} Proceedings},
}

The IAMúsica project

This work was performed in the context of the IAMúsica artistic research project, in collaboration with the amazing Eulàlia Febrer Coll and Joan Lluís Travé Pla. The main goal was to “explore the usage of AI applied to real-time audio analysis to create new tools to support musical practice and bring a new quantitative approach to artistic research”.

We presented our project and conducted a live workshop at the Conservatori Superior de Música de les Illes Balears in September 2022:

At the top of the page you can find the presentation slides (in Catalan and Spanish). In order to conduct the workshop, I prepared a real-time software demonstration that looks like this:

Among other things, it allows to record and load audio files, extract their onset and velocity information, and analyze the velocity distribution and statistics for a given note selection as well as its time-alignment with respect to a grid. We demonstrated this functionality and discussed with music students and scholars possible applications and implications of this technology, which has only recently reached sufficient qualitative and practical levels (and we hope that our work contributed to that).

On a personal note, the IAMúsica project was a key 🎹 opportunity to get acquainted with the full lifecycle of a research project, from conception and redaction of the grant proposal to the final divulgation stage.

It was a true pleasure to collaborate with Eulàlia and Joan, and the project also combined all things that I love: music, computing, research, divulgation, and now we add Mallorca to the list: ⭐ ⭐ ⭐ ⭐ ⭐ experience!