An Immersive Sequencer in Virtual Reality with Natural Interaction and Hybrid Audio Reproduction

Mao J, Nepomuceno L, Ding K, Çamcı A. An Immersive Sequencer in Virtual Reality with Natural Interaction and Hybrid Audio Reproduction. In: AVARIG 2026: Audio for Virtual and Augmented Reality and Immersive Games. Audio Engineering Society; 2026. Paper 10325. Available from: https://aes.org/publications/elibrary-page/?id=23287

@inproceedings{Mao2026_23287,
  author = {Mao, Jiawen and Nepomuceno, Lukas and Ding, Ke and Çamcı, Anil},
  title = {{An Immersive Sequencer in Virtual Reality with Natural Interaction and Hybrid Audio Reproduction}},
  booktitle = {AVARIG 2026: Audio for Virtual and Augmented Reality and Immersive Games},
  note = {Paper 10325},
  year = {2026},
  month = jun,
  publisher = {Audio Engineering Society},
  url = {https://aes.org/publications/elibrary-page/?id=23287}
}

TY  - CPAPER
TI  - An Immersive Sequencer in Virtual Reality with Natural Interaction and Hybrid Audio Reproduction
AU  - Mao, Jiawen
AU  - Nepomuceno, Lukas
AU  - Ding, Ke
AU  - Çamcı, Anil
T2  - AVARIG 2026: Audio for Virtual and Augmented Reality and Immersive Games
M1  - Paper 10325
PY  - 2026
DA  - 2026/06/30
UR  - https://aes.org/publications/elibrary-page/?id=23287
PB  - Audio Engineering Society
LA  - en
AB  - We present Music of the Spheres (MOTS), an immersive virtual reality (VR) sequencer that integrates natural interaction with hybrid spatial audio reproduction for music composition and performance. MOTS enables users to create and manipulate sound objects arranged in a 3D step sequencer surrounding the user. Using hand gestures, users can instantiate, position, and remove sounds, simultaneously composing both temporal and spatial musical structures. The system combines binaural reproduction for private preview in the headset and Ambisonic loudspeaker reproduction for shared listening in audience-oriented experiences. In this paper, we discuss the implementation of MOTS and highlight design considerations for intuitive musical interfaces that are uniquely crafted for VR. We also present the results of a survey of 27 participants at a public exhibition, which indicate positive responses in terms of immersion and usability, as well as a coherent spatial audio experience across the hybrid reproduction system. Finally, we outline future directions, including expanded controls, collaborative functionality, and improved spatial audio rendering.
ER  -