Graphic Score Grammars for End-Users

Stead, Alistair G. and Blackwell, Alan F. and Aaron, Samual

Proceedings of the International Conference on New Interfaces for Musical Expression

We describe a system that allows non-programmers to specify the grammar for a novel graphic score notation of their own design, defining performance notations suitable for drawing in live situations on a surface such as a whiteboard. Thescore can be interpreted via the camera of a smartphone,interactively scanned over the whiteboard to control the parameters of synthesisers implemented in Overtone. The visual grammar of the score, and its correspondence to the sound parameters, can be defined by the user with a simple visual condition-action language. This language can be edited on the touchscreen of an Android phone, allowing the grammar to be modified live in performance situations.Interactive scanning of the score is visible to the audience asa performance interface, with a colour classifier and visual feature recogniser causing the grammar-specified events to be sent using OSC messages via Wi-Fi from the hand-held smartphone to an audio workstation.