Cloud-Enabled Interactive Sound Propagation for Untethered Mixed Reality

Chemistruck M, Storck K, Raghuvanshi N. Cloud-Enabled Interactive Sound Propagation for Untethered Mixed Reality. In: AES Conference: 2020 AES International Conference on Audio for Virtual and Augmented Reality (August 2020). Audio Engineering Society; 2020. Paper 3-5. Available from: https://aes.org/publications/elibrary-page/?id=20882

@inproceedings{Chemistruck2020_20882,
  author = {Chemistruck, Michael and Storck, Kyle and Raghuvanshi, Nikunj},
  title = {{Cloud-Enabled Interactive Sound Propagation for Untethered Mixed Reality}},
  booktitle = {AES Conference: 2020 AES International Conference on Audio for Virtual and Augmented Reality (August 2020)},
  note = {Paper 3-5},
  year = {2020},
  month = aug,
  publisher = {Audio Engineering Society},
  url = {https://aes.org/publications/elibrary-page/?id=20882}
}

TY  - CPAPER
TI  - Cloud-Enabled Interactive Sound Propagation for Untethered Mixed Reality
AU  - Chemistruck, Michael
AU  - Storck, Kyle
AU  - Raghuvanshi, Nikunj
T2  - AES Conference: 2020 AES International Conference on Audio for Virtual and Augmented Reality (August 2020)
M1  - Paper 3-5
PY  - 2020
DA  - 2020/08/06
UR  - https://aes.org/publications/elibrary-page/?id=20882
PB  - Audio Engineering Society
LA  - en
AB  - We describe the first system for physically-based wave acoustics including diffraction effects within a holographic experience shared by multiple untethered devices. Our system scales across standalone mobile-class devices, from a HoloLens to a modern smart phone. Audio propagation in real-world scenes exhibits perceptually salient effects that complement visuals. These include diffraction losses from obstruction, re-direction (“portaling”) of sounds around physical doorways and corners, and reverberation in complex geometries with multiple connected spaces. Such effects are necessary in mixed reality to achieve a sense of presence for virtual people and things within the real world, but have so far been computationally infeasible on mobile devices. We propose a novel cloud-enabled system that enables such immersive audio-visual scenarios on untethered mixed reality devices for the first time.
ER  -