ForTouch : A Wearable Digital Ventriloquized Actor

Sidney S. Fels, Bob Pritchard, and Allison Lenters

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract

We have constructed an easy-to-use portable, wearable gesture-to-speech system based on the Glove-TalkII [1] and GRASSP [2] Digital Ventriloquized Actors (DIVAs). Our new portable system, called a ForTouch, is a specific model of a DIVA and refines the use of a formant speech synthesizer. Using ForTouch, a user can speak using hand gestures mapped to synthetic sound using a mapping function that preserves gesture trajectories. By making ForTouch portable and self-contained, speakers can communicate with others in the community and perform in new music/theatre stage productions. Figure 1 shows one performer using the ForTouch. ForTouch performers also allow us to study the relation between gestures and speech/song production.

Citation

Sidney S. Fels, Bob Pritchard, and Allison Lenters. 2009. ForTouch : A Wearable Digital Ventriloquized Actor. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.1177509 [PDF]

BibTeX Entry

@inproceedings{Fels2009,
 abstract = {We have constructed an easy-to-use portable, wearable gesture-to-speech system based on the Glove-TalkII [1] and GRASSP [2] Digital Ventriloquized Actors (DIVAs). Our new portable system, called a ForTouch, is a specific model of a DIVA and refines the use of a formant speech synthesizer. Using ForTouch, a user can speak using hand gestures mapped to synthetic sound using a mapping function that preserves gesture trajectories. By making ForTouch portable and self-contained, speakers can communicate with others in the community and perform in new music/theatre stage productions. Figure 1 shows one performer using the ForTouch. ForTouch performers also allow us to study the relation between gestures and speech/song production.},
 address = {Pittsburgh, PA, United States},
 author = {Fels, Sidney S. and Pritchard, Bob and Lenters, Allison},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.1177509},
 issn = {2220-4806},
 keywords = {nime09},
 pages = {274--275},
 title = {ForTouch : A Wearable Digital Ventriloquized Actor},
 url = {http://www.nime.org/proceedings/2009/nime2009_274.pdf},
 year = {2009}
}