The Prospects For Eye-Controlled Musical Performance

Hornof, Anthony

Proceedings of the International Conference on New Interfaces for Musical Expression

Although new sensor devices and data streams are increasingly used for musical expression, and although eye-tracking devices have become increasingly cost-effective and prevalent in research and as a means of communication for people with severe motor impairments, eye-controlled musical expression nonetheless remains somewhat elusive and minimally explored. This paper (a) identifies a number of fundamental human eye movement capabilities and constraints which determine in part what can and cannot be musically expressed with eye movements, (b) reviews prior work on eye-controlled musical expression, and (c) analyzes and provides a taxonomy of what has been done, and what will need to be addressed in future eye-controlled musical instruments. The fundamental human constraints and processes that govern eye movements create a challenge for eye-controlled music in that the instrument needs to be designed to motivate or at least permit specific unique visual goals, each of which when accomplished must then be mapped, using the eye tracker and some sort of sound generator, to different musical outcomes. The control of the musical instrument is less direct than if it were played with muscles that can be controlled in a more direct manner, such as the muscles in the hands.