GestoLumina: Gesture interpreted Light, Sound and Haptics. Towards a Framework for Universal Music Design

Nate Hergert, Dillon Simeone, Doga Cavdir, Duncan MacConnell, Shawn Trail, and Myles de Bastion

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract:

Introducing GestoLumina (GeLu), a novel interface by the Universal Music Design (UMD) team at CymaSpace, whose mission is to enhance arts and culture accessibility for the Deaf and Hard of Hearing (DHH). UMD is a program that applies principals taken from the "Universal Design" concept towards music accessibility and inclusion for the DHH. The primary goal of UMD is to create experiential, educational pathways for the DHH to interact with music in a meaningful way. GeLu is the first UMD interface in the framework and is modular in its feature set allowing the user to choose, based on use case, between visual and/or haptic feedback along with gestural sensing. GeLu combines two bracelets: a ring system with gesture sensing and haptic feedback; and an audio-reactive LED color feedback array. The ring system captures tactile input through sensors and provides haptic feedback on the fingertips. The audio-reactive LED color feedback bracelet visualizes audio data, creating dynamic visualizations synchronized with sound.

Citation:

Nate Hergert, Dillon Simeone, Doga Cavdir, Duncan MacConnell, Shawn Trail, and Myles de Bastion. 2024. GestoLumina: Gesture interpreted Light, Sound and Haptics. Towards a Framework for Universal Music Design. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.13904794

BibTeX Entry:

  @article{nime2024_15,
 abstract = {Introducing GestoLumina (GeLu), a novel interface by the Universal Music Design (UMD) team at CymaSpace, whose mission is to enhance arts and culture accessibility for the Deaf and Hard of Hearing (DHH). UMD is a program that applies principals taken from the "Universal Design" concept towards music accessibility and inclusion for the DHH. The primary goal of UMD is to create experiential, educational pathways for the DHH to interact with music in a meaningful way. GeLu is the first UMD interface in the framework and is modular in its feature set allowing the user to choose, based on use case, between visual and/or haptic feedback along with gestural sensing. GeLu combines two bracelets: a ring system with gesture sensing and haptic feedback; and an audio-reactive LED color feedback array. The ring system captures tactile input through sensors and provides haptic feedback on the fingertips. The audio-reactive LED color feedback bracelet visualizes audio data, creating dynamic visualizations synchronized with sound. },
 address = {Utrecht, Netherlands},
 articleno = {15},
 author = {Nate Hergert and Dillon Simeone and Doga Cavdir and Duncan MacConnell and Shawn Trail and Myles de Bastion},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.13904794},
 editor = {S M Astrid Bin and Courtney N. Reed},
 issn = {2220-4806},
 month = {September},
 numpages = {4},
 pages = {100--103},
 presentation-video = {},
 title = {GestoLumina: Gesture interpreted Light, Sound and Haptics. Towards a Framework for Universal Music Design},
 track = {Papers},
 url = {http://nime.org/proceedings/2024/nime2024_15.pdf},
 year = {2024}
}