Augmented Piano Performance using a Depth Camera

Yang, Qi and Essl, Georg

Proceedings of the International Conference on New Interfaces for Musical Expression

We augment the piano keyboard with a 3D gesture space using Microsoft Kinect for sensing and top-down projection for visual feedback. This interface provides multi-axial gesture controls to enable continuous adjustments to multiple acoustic parameters such as those on the typical digital synthesizers. We believe that using gesture control is more visceral and aesthetically pleasing, especially during concert performance where the visibility of the performer’s action is important. Our system can also be used for other types of gesture interaction as well as for pedagogical applications.