An Interactive System for Gesture-Controlled Vocal Improvisation with Somax2: Cultural Narrative in Minotaur's Lament

This paper presents the design and implementation of a new system for human-computer interactive performances, integrating gesture recognition for live vocal improvisation with Somax2, particularly within cultural contexts. Using the MediaPipe framework in Python for real-time hand tracking, performer gestures are mapped to live signals, which are transmitted to the Max/MSP environment with the Open Sound Control (OSC) protocol. The received signals are mapped to parameters for controlling live vocal effects, which influence Somax2's generative musical output. We then demonstrate the system's usage and performance workflow with Minotaur's Lament, an improvisation inspired by Greek mythology and Ancient Greek musical forms. Mapping gestures to pitch shift, we find that the proposed framework enables enhanced vocal expressivity and Somax2 musical variability. As an additional control layer for the Somax2 architecture, the system widens the improviser's creative flexibility, with potential for further applications in culturally-embedded interactive performances and computer music interfaces. The paper provides an overview of the system architecture, technical implementation of the performance, as well as the artistic background and intent. A video demonstrating the usage our system in performance: Minotaur's Lament All source code is available at: https://github.com/pinganyao/minotaurs-lament