Musical Interface to Audiovisual Corpora of Arbitrary Instruments

Joachim Goßmann, and Max Neupert

Proceedings of the International Conference on New Interfaces for Musical Expression

• Year: 2014
• Location: London, United Kingdom
• Pages: 151–154
• DOI: 10.5281/zenodo.1178772 (Link to paper)
• PDF link

Abstract:

We present an instrument for audio-visual performance that allows to recombine sounds from a collection of sampled media through concatenative synthesis. A three-dimensional distribution derived from feature-analysis becomes accessible through a theremin-inspired interface, allowing the player to shift from exploration and intuitive navigation toward embodied performance on a granular level. In our example we illustrate this concept by using the audiovisual recording of an instrumental performance as a source. Our system provides an alternative interface to the musical instrument's audiovisual corpus: as the instrument's sound and behavior is accessed in ways that are not possible on the instrument itself, the resulting non-linear playback of the grains generates an instant remix in a cut-up aesthetic. The presented instrument is a human-computer interface that employs the structural outcome of machine analysis accessing audiovisual corpora in the context of a musical performance.

Citation:

BibTeX Entry:

  @inproceedings{mneupert2014,
 abstract = {We present an instrument for audio-visual performance that allows to recombine sounds from a collection of sampled media through concatenative synthesis. A three-dimensional distribution derived from feature-analysis becomes accessible through a theremin-inspired interface, allowing the player to shift from exploration and intuitive navigation toward embodied performance on a granular level. In our example we illustrate this concept by using the audiovisual recording of an instrumental performance as a source. Our system provides an alternative interface to the musical instrument's audiovisual corpus: as the instrument's sound and behavior is accessed in ways that are not possible on the instrument itself, the resulting non-linear playback of the grains generates an instant remix in a cut-up aesthetic. The presented instrument is a human-computer interface that employs the structural outcome of machine analysis accessing audiovisual corpora in the context of a musical performance.},
 address = {London, United Kingdom},
 author = {Joachim Go{\ss}mann and Max Neupert},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.1178772},
 issn = {2220-4806},
 month = {June},
 pages = {151--154},
 publisher = {Goldsmiths, University of London},
 title = {Musical Interface to Audiovisual Corpora of Arbitrary Instruments},
 url = {http://www.nime.org/proceedings/2014/nime2014_296.pdf},
 year = {2014}
}