Audiovisual Congruence and Localization Performance in Virtual Reality: 3D Loudspeaker Model vs. Human Avatar

Hofmann A, Meyer-Kahlen N, Schlecht SJ, Lokki T. Audiovisual Congruence and Localization Performance in Virtual Reality: 3D Loudspeaker Model vs. Human Avatar. J Audio Eng Soc. 2024;72(10):679-690. doi:10.17743/jaes.2022.0162

@article{Hofmann2024_22771,
  author = {Hofmann, Anja and Meyer-Kahlen, Nils and Schlecht, Sebastian J. and Lokki, Tapio},
  title = {{Audiovisual Congruence and Localization Performance in Virtual Reality: 3D Loudspeaker Model vs. Human Avatar}},
  journal = {Journal of the Audio Engineering Society},
  volume = {72},
  number = {10},
  pages = {679--690},
  year = {2024},
  month = oct,
  publisher = {Audio Engineering Society},
  doi = {10.17743/jaes.2022.0162},
  url = {https://doi.org/10.17743/jaes.2022.0162}
}

TY  - JOUR
TI  - Audiovisual Congruence and Localization Performance in Virtual Reality: 3D Loudspeaker Model vs. Human Avatar
AU  - Hofmann, Anja
AU  - Meyer-Kahlen, Nils
AU  - Schlecht, Sebastian J.
AU  - Lokki, Tapio
T2  - Journal of the Audio Engineering Society
J2  - J. Audio Eng. Soc.
VL  - 72
IS  - 10
SP  - 679
EP  - 690
PY  - 2024
DA  - 2024/10/15
DO  - 10.17743/jaes.2022.0162
UR  - https://doi.org/10.17743/jaes.2022.0162
PB  - Audio Engineering Society
LA  - en
AB  - This paper investigates audiovisual congruence in virtual reality with both horizontal and vertical offsets between audio and visual rendering. Audiovisual congruence and localization errors are assessed using loudspeaker playback and nonindividualized headphone rendering. To account for the influence of different types of visual information on congruence, presentations of a loudspeaker model and 3D human avatar were compared. Therefore, a new dataset of audiovisual speech was recorded. Results show that human avatar rendering increases perceived congruence, and experienced listeners have an increased tendency to respond with “incongruent” when a loudspeaker model is shown but not when the human avatar is presented. Moreover, a correlation is found between localization precision and audiovisual congruence for horizontally offset stimuli and avatar presentation. For vertical offsets, the angular range of congruence is generally large, and localization errors are high, so no correlation can be observed between the two. The paper contributes congruence ranges for audiovisual speech in virtual reality, which also has implications for augmented reality telepresence use.
ER  -