Effects of Spatial Audiovisual Incongruency on Mental Workload and Task Performance in Immersive VR

Sall C, Roginska A. Effects of Spatial Audiovisual Incongruency on Mental Workload and Task Performance in Immersive VR. In: AVARIG 2026: Audio for Virtual and Augmented Reality and Immersive Games. Audio Engineering Society; 2026. Paper 474. Available from: https://aes.org/publications/elibrary-page/?id=23366

@inproceedings{Sall2026_23366,
  author = {Sall, Catherine and Roginska, Agnieszka},
  title = {{Effects of Spatial Audiovisual Incongruency on Mental Workload and Task Performance in Immersive VR}},
  booktitle = {AVARIG 2026: Audio for Virtual and Augmented Reality and Immersive Games},
  note = {Paper 474},
  year = {2026},
  month = jun,
  publisher = {Audio Engineering Society},
  url = {https://aes.org/publications/elibrary-page/?id=23366}
}

TY  - CPAPER
TI  - Effects of Spatial Audiovisual Incongruency on Mental Workload and Task Performance in Immersive VR
AU  - Sall, Catherine
AU  - Roginska, Agnieszka
T2  - AVARIG 2026: Audio for Virtual and Augmented Reality and Immersive Games
M1  - Paper 474
PY  - 2026
DA  - 2026/06/30
UR  - https://aes.org/publications/elibrary-page/?id=23366
PB  - Audio Engineering Society
LA  - en
AB  - Virtual reality (VR) has emerged as a promising medium for high-stakes training, yet its predominantly visual design places disproportionate demands on attentional resources, limiting cognitive capacity for other task-relevant information. Spatial audio cues exploit the auditory channel to redistribute this perceptual load, with demonstrated improvements in reaction time, search efficiency, and situational awareness. However, when audio cues are spatially misaligned with visual targets, they can degrade performance. The behavioral and cognitive costs of such incongruency remain underexplored. This pilot study examined how audiovisual spatial incongruency affects task performance and subjective mental workload in an immersive VR environment. Seventeen participants completed a within-subjects search-and-respond task across congruent and incongruent audiovisual conditions with 8, 10, or 12 potential target locations. Reaction time, timeout rate, head rotation, and subjective ratings of mental demand and frustration were measured across 10 trials per participant. Results revealed consistent and significant costs of audiovisual incongruency. Participants responded approximately 10.5% slower, were 5 to 6 times more likely to time out, and reported higher mental demand and frustration under conditions of audiovisual spatial misalignment. The effect of audiovisual incongruency on workload was not significantly mediated by behavioral performance, indicating that spatial conflict carries a direct subjective cost independent of task outcomes. These findings inform spatial audio design for immersive training systems and motivate further investigation into tolerance thresholds for audiovisual misalignment and their effects on learning outcomes.
ER  -