A System Architecture for Semantically Informed Rendering of Object-Based Audio

Franck A, Francombe J, Woodcock J, Hughes R, Coleman P, Menzies D, Cox TJ, Jackson PJB, Fazi FM. A System Architecture for Semantically Informed Rendering of Object-Based Audio. J Audio Eng Soc. 2019;67(7/8):498-509. doi:10.17743/jaes.2019.0025

@article{Franck2019_20488,
  author = {Franck, Andreas and Francombe, Jon and Woodcock, James and Hughes, Richard and Coleman, Philip and Menzies, Dylan and Cox, Trevor J. and Jackson, Philip J.B. and Fazi, Filippo Maria},
  title = {{A System Architecture for Semantically Informed Rendering of Object-Based Audio}},
  journal = {Journal of the Audio Engineering Society},
  volume = {67},
  number = {7/8},
  pages = {498--509},
  year = {2019},
  month = jul,
  publisher = {Audio Engineering Society},
  doi = {10.17743/jaes.2019.0025},
  url = {https://doi.org/10.17743/jaes.2019.0025}
}

TY  - JOUR
TI  - A System Architecture for Semantically Informed Rendering of Object-Based Audio
AU  - Franck, Andreas
AU  - Francombe, Jon
AU  - Woodcock, James
AU  - Hughes, Richard
AU  - Coleman, Philip
AU  - Menzies, Dylan
AU  - Cox, Trevor J.
AU  - Jackson, Philip J.B.
AU  - Fazi, Filippo Maria
T2  - Journal of the Audio Engineering Society
J2  - J. Audio Eng. Soc.
VL  - 67
IS  - 7/8
SP  - 498
EP  - 509
PY  - 2019
DA  - 2019/07/06
DO  - 10.17743/jaes.2019.0025
UR  - https://doi.org/10.17743/jaes.2019.0025
PB  - Audio Engineering Society
LA  - en
AB  - Object-based audio promises format-agnostic reproduction and extensive personalization of spatial audio content. However, in practical listening scenarios, such as in consumer audio, ideal reproduction is typically not possible. To maximize the quality of listening experience, a different approach is required, for example modifications of metadata to adjust for the reproduction layout or personalization choices. This paper proposes a novel system architecture for semantically informed rendering (SIR), that combines object audio rendering with high-level processing of object metadata. In many cases, this processing uses novel, advanced metadata describing the objects to optimally adjust the audio scene to the reproduction system or listener preferences. The proposed system is evaluated with several adaptation strategies, including semantically motivated downmix to layouts with few loudspeakers, manipulation of perceptual attributes, perceptual reverberation compensation, and orchestration of mobile devices for immersive reproduction. These examples demonstrate how SIR can significantly improve the media experience and provide advanced personalization controls, for example by maintaining smooth object trajectories on systems with few loudspeakers, or providing personalized envelopment levels. An example implementation of the proposed system architecture is described and provided as an open, extensible software framework that combines object-based audio rendering and high-level processing of advanced object metadata.
ER  -