A Dialogue-Enhanced Companion System for Tracking Simultaneous Interactions in an Immersive Environment

Chang A, Briggs S, Chabot S, Braasch J. A Dialogue-Enhanced Companion System for Tracking Simultaneous Interactions in an Immersive Environment. In: AES Convention 155. Audio Engineering Society; 2023. Paper 136. Available from: https://aes.org/publications/elibrary-page/?id=22290

@inproceedings{Chang2023_22290,
  author = {Chang, Albert and Briggs, Shannon and Chabot, Sam and Braasch, Jonas},
  title = {{A Dialogue-Enhanced Companion System for Tracking Simultaneous Interactions in an Immersive Environment}},
  booktitle = {AES Convention 155},
  note = {Paper 136},
  year = {2023},
  month = oct,
  publisher = {Audio Engineering Society},
  url = {https://aes.org/publications/elibrary-page/?id=22290}
}

TY  - CPAPER
TI  - A Dialogue-Enhanced Companion System for Tracking Simultaneous Interactions in an Immersive Environment
AU  - Chang, Albert
AU  - Briggs, Shannon
AU  - Chabot, Sam
AU  - Braasch, Jonas
T2  - AES Convention 155
M1  - Paper 136
PY  - 2023
DA  - 2023/10/06
UR  - https://aes.org/publications/elibrary-page/?id=22290
PB  - Audio Engineering Society
LA  - en
AB  - As recent improvements in task-oriented and open-domain systems - paired with deep learning technologies - have enhanced the potential for practical applications across many fields, dialogue systems have become a prominent study medium. Multimodal dialogue systems deployed in interactive settings to provide users with an immersive experience are one such vein of applications. This idea presents a potential interactive system as a medium that combines spatial awareness with a multi-modal, immersive dialogue system to provide an additional layer of immersion. The system makes use of an array of audio/visual sensors to follow individuals as they move about the interactive space. It responds contextually based on the application and information domain, such as by displaying and sonifying conversational agents at precise spatial locations. A full scale system has been developed and was tested in a pilot study for a university-level Mandarin Chinese class, while a down-scaled version of this system is in development to investigate benefits and behaviors for its use in group collaboration.
ER  -