ESP: A Driving Interface for Expression Synthesis

Chew, Elaine and Francois, Alexander R. and Liu, Jie and Yang, Aaron

Proceedings of the International Conference on New Interfaces for Musical Expression

In the Expression Synthesis Project (ESP), we propose adriving interface for expression synthesis. ESP aims toprovide a compelling metaphor for expressive performance soas to make high-level expressive decisions accessible to nonexperts. In ESP, the user drives a car on a virtual road thatrepresents the music with its twists and turns; and makesdecisions on how to traverse each part of the road. The driver’sdecisions affect in real-time the rendering of the piece. Thepedals and wheel provide a tactile interface for controlling thecar dynamics and musical expression, while the displayportrays a first person view of the road and dashboard from thedriver’s seat. This game-like interface allows non-experts tocreate expressive renderings of existing music without havingto master an instrument, and allows expert musicians toexperiment with expressive choice without having to firstmaster the notes of the piece. The prototype system has beentested and refined in numerous demonstrations. This paperpresents the concepts underlying the ESP system and thearchitectural design and implementation of a prototype.