A Novel Face-tracking Mouth Controller and its Application to Interacting with Bioacoustic Models

Gamhewage C. de Silva, Tamara Smyth, and Michael J. Lyons

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract:

We describe a simple, computationally light, real-time system for tracking the lower face and extracting informationabout the shape of the open mouth from a video sequence.The system allows unencumbered control of audio synthesismodules by action of the mouth. We report work in progressto use the mouth controller to interact with a physical modelof sound production by the avian syrinx.

Citation:

Gamhewage C. de Silva, Tamara Smyth, and Michael J. Lyons. 2004. A Novel Face-tracking Mouth Controller and its Application to Interacting with Bioacoustic Models. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.1176667

BibTeX Entry:

@inproceedings{Silva2004,
 abstract = {We describe a simple, computationally light, real-time system for tracking the lower face and extracting informationabout the shape of the open mouth from a video sequence.The system allows unencumbered control of audio synthesismodules by action of the mouth. We report work in progressto use the mouth controller to interact with a physical modelof sound production by the avian syrinx.},
 address = {Hamamatsu, Japan},
 author = {de Silva, Gamhewage C. and Smyth, Tamara and Lyons, Michael J.},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.1176667},
 issn = {2220-4806},
 keywords = {Mouth Controller, Face Tracking, Bioacoustics},
 pages = {169--172},
 title = {A Novel Face-tracking Mouth Controller and its Application to Interacting with Bioacoustic Models},
 url = {http://www.nime.org/proceedings/2004/nime2004_169.pdf},
 year = {2004}
}