Using Music to Interact with a Virtual Character

Taylor, Robyn and Torres, Daniel and Boulanger, Pierre

Proceedings of the International Conference on New Interfaces for Musical Expression

We present a real-time system which allows musicians tointeract with synthetic virtual characters as they perform.Using Max/MSP to parameterize keyboard and vocal input, meaningful features (pitch, amplitude, chord information, and vocal timbre) are extracted from live performancein real-time. These extracted musical features are thenmapped to character behaviour in such a way that the musician’s performance elicits a response from the virtual character. The system uses the ANIMUS framework to generatebelievable character expressions. Experimental results arepresented for simple characters.