Spatial Auditory-Visual Integration: The Case of Binaural Sound on a Smartphone

Moreira J, Gros L, Nicol R, Viaud-Delmon I. Spatial Auditory-Visual Integration: The Case of Binaural Sound on a Smartphone. In: AES Convention 145. Audio Engineering Society; 2018. Paper 10130. Available from: https://aes.org/publications/elibrary-page/?id=19856

@inproceedings{Moreira2018_19856,
  author = {Moreira, Julian and Gros, Laetitia and Nicol, Rozenn and Viaud-Delmon, Isabelle},
  title = {{Spatial Auditory-Visual Integration: The Case of Binaural Sound on a Smartphone}},
  booktitle = {AES Convention 145},
  note = {Paper 10130},
  year = {2018},
  month = oct,
  publisher = {Audio Engineering Society},
  url = {https://aes.org/publications/elibrary-page/?id=19856}
}

TY  - CPAPER
TI  - Spatial Auditory-Visual Integration: The Case of Binaural Sound on a Smartphone
AU  - Moreira, Julian
AU  - Gros, Laetitia
AU  - Nicol, Rozenn
AU  - Viaud-Delmon, Isabelle
T2  - AES Convention 145
M1  - Paper 10130
PY  - 2018
DA  - 2018/10/06
UR  - https://aes.org/publications/elibrary-page/?id=19856
PB  - Audio Engineering Society
LA  - en
AB  - Binaural rendering is a technology for spatialized sound that can be advantageously coupled with the visual of a mobile phone. By rendering the auditory scene out of the screen, all around the user, it is a potentially powerful tool of immersion. However, this audio-visual association may lead to specific perception artifacts. One of them is the ventriloquist effect, i.e., the perception of a sound and an image as they come from the same location, while they are actually at different places. We investigate the conditions of this effect to occur using an experimental method called Point of Subjective Spatial Alignment (PSSA). Given the position of a visual stimulus, we determine the integration window, i.e., the range of locations in the horizontal plane where auditory stimulus is perceived as matching the visual stimulus location. Several parameters are varied: semantic type of the stimuli (neutral or meaningful) and sound elevation (same elevation as the visual or above subject’s head). Results reveal the existence of an integration window in all cases. But, surprisingly, the sound is attracted by the visual as located in the virtual scene, rather than its real location on screen. We interpret it as a mark of immersion. Besides, we observe that integration window is not altered by elevation, provided that stimuli are semantically meaningful.
ER  -