A Self-Organizing Gesture Map for a Voice-Controlled Instrument Interface

Fasciani, Stefano and Wyse, Lonce

Proceedings of the International Conference on New Interfaces for Musical Expression

Mapping gestures to digital musical instrument parameters is not trivial when the dimensionality of the sensor-captured data is high and the model relating the gesture to sensor data is unknown. In these cases, a front-end processing system for extracting gestural information embedded in the sensor data is essential. In this paper we propose an unsupervised offline method that learns how to reduce and map the gestural data to a generic instrument parameter control space. We make an unconventional use of the Self-Organizing Maps to obtain only a geometrical transformation of the gestural data, while dimensionality reduction is handled separately. We introduce a novel training procedure to overcome two main Self-Organizing Maps limitations which otherwise corrupt the interface usability. As evaluation, we apply this method to our existing Voice-Controlled Interface for musical instruments, obtaining sensible usability improvements.