Automatic Acoustical Material Estimation for Room-Specific Early Reflections in Auditory Augmented Reality (AAR) Application

Sato R, Lee K, Izumi Y, Kim S. Automatic Acoustical Material Estimation for Room-Specific Early Reflections in Auditory Augmented Reality (AAR) Application. In: AES Conference: AES 2024 International Audio for Games Conference. Audio Engineering Society; 2024. Paper 15. Available from: https://aes.org/publications/elibrary-page/?id=22404

@inproceedings{Sato2024_22404,
  author = {Sato, Rai and Lee, Kangeun and Izumi, Yuto and Kim, Sungyoung},
  title = {{Automatic Acoustical Material Estimation for Room-Specific Early Reflections in Auditory Augmented Reality (AAR) Application}},
  booktitle = {AES Conference: AES 2024 International Audio for Games Conference},
  note = {Paper 15},
  year = {2024},
  month = apr,
  publisher = {Audio Engineering Society},
  url = {https://aes.org/publications/elibrary-page/?id=22404}
}

TY  - CPAPER
TI  - Automatic Acoustical Material Estimation for Room-Specific Early Reflections in Auditory Augmented Reality (AAR) Application
AU  - Sato, Rai
AU  - Lee, Kangeun
AU  - Izumi, Yuto
AU  - Kim, Sungyoung
T2  - AES Conference: AES 2024 International Audio for Games Conference
M1  - Paper 15
PY  - 2024
DA  - 2024/04/06
UR  - https://aes.org/publications/elibrary-page/?id=22404
PB  - Audio Engineering Society
LA  - en
AB  - Auditory Augmented Reality (AAR) is an emerging technology that enhances user experiences by overlaying spatially appropriate sounds onto a real environment and gives the impression of virtual sounds coexisting with the physical world. A major challenge in achieving a plausible real-time AAR experience is to seamlessly align virtual soundscapes with real-world environments. To address this, we previously introduced the Augmented reality Room Acoustic Estimator (ARAE). However, the earlier versions of the ARAE encountered difficulties in appropriately replicating the room acoustic behavior due to inadequate rendering of early reflections. To overcome these challenges, we implemented two innovative techniques: mesh segmentation to identify dominant acoustic elements and material estimation through deep learning. Our pilot measurement demonstrated that the proposed method better replicates the sound behavior of the physical environment than the one without material estimation in terms of its frequency characteristics, although some discrepancies were also noted. Future developments will concentrate on customizing the material estimation datasets with the intention of inferring real-world acoustic conditions and optimizing the proposed method through subjective evaluations.
ER  -