Pointing Fingers: Using Multiple Direct Interactions with Visual Objects to Perform Music

Couturier, Jean-Michel and Arfib, Daniel

Proceedings of the International Conference on New Interfaces for Musical Expression

In this paper, we describe a new interface for musicalperformance, using the interaction with a graphical userinterface in a powerful manner: the user directly touches ascreen where graphical objects are displayed and can useseveral fingers simultaneously to interact with the objects. Theconcept of this interface is based on the superposition of thegesture spatial place and the visual feedback spatial place; i tgives the impression that the graphical objects are real. Thisconcept enables a huge freedom in designing interfaces. Thegesture device we have created gives the position of fourfingertips using 3D sensors and the data is performed in theMax/MSP environment. We have realized two practicalexamples of musical use of such a device, using PhotosonicSynthesis and Scanned Synthesis.