Designing a new virtual reality interface for interacting with audio spatialization backends

Zétény Nagy, Kristin Carlson, and Greg J Corness

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract:

xrOSC presents an alternative spatialization tool for composers working in 3D sound formats. We developed this tool to have an easier learning curve, and a low barrier of entry. xrOSC is an isotonic mixed reality controller interface with six degrees of freedom and direct absolute input designed for standalone extended reality devices. By utilizing hand tracking and gestural control, we can enable more natural and intuitive positioning of sound sources in 3D audio spatialization of the external composed space, as opposed to using spatialization tools on a traditional 2D display with mouse/keyboard input methods. This software is designed specifically as a control method, as there is no processing done on the device. xrOSC simply sends control messages over the network to a spatialization backend using the Open Sound Control protocol. This paper discusses the design process and methodological considerations in order to develop a tool for composers to more easily create spatial-considered musical compositions.

Citation:

Zétény Nagy, Kristin Carlson, and Greg J Corness. 2024. Designing a new virtual reality interface for interacting with audio spatialization backends. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.13904830

BibTeX Entry:

  @article{nime2024_31,
 abstract = {xrOSC presents an alternative spatialization tool for composers working in 3D sound formats. We developed this tool to have an easier learning curve, and a low barrier of entry. xrOSC is an isotonic mixed reality controller interface with six degrees of freedom and direct absolute input designed for standalone extended reality devices. By utilizing hand tracking and gestural control, we can enable more natural and intuitive positioning of sound sources in 3D audio spatialization of the external composed space, as opposed to using spatialization tools on a traditional 2D display with mouse/keyboard input methods. This software is designed specifically as a control method, as there is no processing done on the device. xrOSC simply sends control messages over the network to a spatialization backend using the Open Sound Control protocol. This paper discusses the design process and methodological considerations in order to develop a tool for composers to more easily create spatial-considered musical compositions.},
 address = {Utrecht, Netherlands},
 articleno = {31},
 author = {Zétény Nagy and Kristin Carlson and Greg J Corness},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.13904830},
 editor = {S M Astrid Bin and Courtney N. Reed},
 issn = {2220-4806},
 month = {September},
 numpages = {4},
 pages = {194--197},
 presentation-video = {},
 title = {Designing a new virtual reality interface for interacting with audio spatialization backends},
 track = {Papers},
 url = {http://nime.org/proceedings/2024/nime2024_31.pdf},
 year = {2024}
}