Exploring Visuo-Auditory Coherence in Virtual Acoustic Environments: A Perceptual Study

Fargeot S, Aramaki M, Kronland-Martinet R. Exploring Visuo-Auditory Coherence in Virtual Acoustic Environments: A Perceptual Study. In: AES 2024 International Conference on Audio for Virtual and Augmented Reality. Audio Engineering Society; 2024. Paper 21. Available from: https://aes.org/publications/elibrary-page/?id=22670

@inproceedings{Fargeot2024_22670,
  author = {Fargeot, Simon and Aramaki, Mitsuko and Kronland-Martinet, Richard},
  title = {{Exploring Visuo-Auditory Coherence in Virtual Acoustic Environments: A Perceptual Study}},
  booktitle = {AES 2024 International Conference on Audio for Virtual and Augmented Reality},
  note = {Paper 21},
  year = {2024},
  month = aug,
  publisher = {Audio Engineering Society},
  url = {https://aes.org/publications/elibrary-page/?id=22670}
}

TY  - CPAPER
TI  - Exploring Visuo-Auditory Coherence in Virtual Acoustic Environments: A Perceptual Study
AU  - Fargeot, Simon
AU  - Aramaki, Mitsuko
AU  - Kronland-Martinet, Richard
T2  - AES 2024 International Conference on Audio for Virtual and Augmented Reality
M1  - Paper 21
PY  - 2024
DA  - 2024/08/05
UR  - https://aes.org/publications/elibrary-page/?id=22670
PB  - Audio Engineering Society
LA  - en
AB  - The study presented here explores the perceived visual-auditory (VA) coherence between acoustic environments and their visual representations, addressing two main questions: are we able to associate room acoustics with their own visual representations, and does the type of visual rendering affect this judgment? A virtual reality (VR) experiment was conducted with five acoustic environments auralized through a Higher Order Ambisonics (HOA) multichannel system, each presented along with three types of visual representations: 360 photos, textured point clouds, and simple 3D models, rendered through a VR headset. For each VA condition, participants were asked to answer a simple question: "Do you think the sound you are hearing was produced in the environment you are seeing? Y/N." Results indicated that congruent situations were not consistently perceived as coherent, and the type of visual rendering did not influence judgments. These findings have implications for applications in cinema, video games, architectural simulation, and virtual reality.
ER  -