MusicMapper: Interactive 2D representations of music samples for in-browser remixing and exploration

Benjamin, Ethan and Altosaar, Jaan

Proceedings of the International Conference on New Interfaces for Musical Expression

Much of the challenge and appeal in remixing music comes from manipulating samples. Typically, identifying distinct samples of a song requires expertise in music production software. Additionally, it is difficult to visualize similarities and differences between all samples of a song simultaneously and use this to select samples. MusicMapper is a web application that allows nonexpert users to find and visualize distinctive samples from a song without any manual intervention, and enables remixing by having users play back clusterings of such samples. This is accomplished by splitting audio from the Soundcloud API into appropriately-sized spectrograms, and applying the t-SNE algorithm to visualize these spectrograms in two dimensions. Next, we apply k-means to guide the user’s eye toward related clusters and set k=26 to enable playback of the clusters by pressing keys on an ordinary keyboard. We present the source code (https://github.com/fatsmcgee/MusicMappr) and a demo video (http://youtu.be/mvD6e1uiO8k) of the MusicMapper web application that can be run in most modern browsers.