Diffy

Jordan Shier, and Xiaowan Yi

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract

Diffy is a duo music project comprising a drummer and a sound designer, connected by a set of machine learning-based sound design agents. In this project, we explore and juxtapose a set of three machine learning-based techniques for manipulating the timbral qualities of percussion instruments in real-time with low-latency. These techniques include a neural audio synthesizer trained on non-percussive material, a timbre remapping 808 drum synthesizer, and a modular synthesizer controlled by a neural network. Each sound design agent operates on different modes of timbral understanding -- reacting to the drum performance based on this understanding, and suggesting sonic transformations. Sonic negotiations between the human sound designer and the sound-design agent are relayed back to the drummer, creating a feedback loop that shapes a structured improvisation.

Citation

Jordan Shier, and Xiaowan Yi. 2025. Diffy. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.17801126 [PDF]

BibTeX Entry

@inproceedings{nime2025_music_26,
 abstract = {Diffy is a duo music project comprising a drummer and a sound designer, connected by a set of machine learning-based sound design agents. In this project, we explore and juxtapose a set of three machine learning-based techniques for manipulating the timbral qualities of percussion instruments in real-time with low-latency. These techniques include a neural audio synthesizer trained on non-percussive material, a timbre remapping 808 drum synthesizer, and a modular synthesizer controlled by a neural network. Each sound design agent operates on different modes of timbral understanding -- reacting to the drum performance based on this understanding, and suggesting sonic transformations. Sonic negotiations between the human sound designer and the sound-design agent are relayed back to the drummer, creating a feedback loop that shapes a structured improvisation.},
 address = {Canberra, Australia},
 articleno = {26},
 author = {Jordan Shier and Xiaowan Yi},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.17801126},
 editor = {Sophie Rose and Jos Mulder and Nicole Carroll},
 issn = {2220-4806},
 month = {June},
 note = {Live Performance},
 numpages = {4},
 pages = {103--106},
 presentation-video = {https://youtu.be/mwq-rSJN048},
 title = {Diffy},
 track = {Music},
 url = {http://nime.org/proceedings/2025/nime2025_music_26.pdf},
 year = {2025}
}