claVision: Visual Automatic Piano Music Transcription

Akbari, Mohammad and Cheng, Howard

Proceedings of the International Conference on New Interfaces for Musical Expression

One important problem in Musical Information Retrieval is Automatic Music Transcription, which is an automated conversion process from played music to a symbolic notation such as sheet music. Since the accuracy of previous audio-based transcription systems is not satisfactory, we propose an innovative visual-based automatic music transcription system named claVision to perform piano music transcription. Instead of processing the music audio, the system performs the transcription only from the video performance captured by a camera mounted over the piano keyboard. claVision can be used as a transcription tool, but it also has other applications such as music education. The claVision software has a very high accuracy (over 95%) and a very low latency in real-time music transcription, even under different illumination conditions.