Synthesis of Complex Sound Scenes with Transformation of Recorded Spatial Sound in Virtual Reality

Pihlajamaki T, Pulkki V. Synthesis of Complex Sound Scenes with Transformation of Recorded Spatial Sound in Virtual Reality. J Audio Eng Soc. 2015;63(7/8):542-551. doi:10.17743/jaes.2015.0059

@article{Pihlajamaki2015_17840,
  author = {Pihlajamaki, Tapani and Pulkki, Ville},
  title = {{Synthesis of Complex Sound Scenes with Transformation of Recorded Spatial Sound in Virtual Reality}},
  journal = {Journal of the Audio Engineering Society},
  volume = {63},
  number = {7/8},
  pages = {542--551},
  year = {2015},
  month = jul,
  publisher = {Audio Engineering Society},
  doi = {10.17743/jaes.2015.0059},
  url = {https://doi.org/10.17743/jaes.2015.0059}
}

TY  - JOUR
TI  - Synthesis of Complex Sound Scenes with Transformation of Recorded Spatial Sound in Virtual Reality
AU  - Pihlajamaki, Tapani
AU  - Pulkki, Ville
T2  - Journal of the Audio Engineering Society
J2  - J. Audio Eng. Soc.
VL  - 63
IS  - 7/8
SP  - 542
EP  - 551
PY  - 2015
DA  - 2015/07/06
DO  - 10.17743/jaes.2015.0059
UR  - https://doi.org/10.17743/jaes.2015.0059
PB  - Audio Engineering Society
LA  - en
AB  - When constructing virtual realities, the task of the sound designer is usually to create a suitable immersive sound scene to fit the situation in the virtual world. This paper presented a real-time method (spatially modified synthesis, i.e., SpaMoS) for modifying the reproduction of a spatial sound recording based on the listener movement in the virtual reality to provide the perception of believable movement inside the recorded sound scene. The method decomposes the omnidirectional signal of a B-format recording into granular virtual omnidirectional sources and then places them on an arbitrary surface defining a convex hull. The placement is based on an energetic analysis performed on the B-format signal. This synthesis is complemented with a separate diffuse synthesis that ideally reproduces the ambience and reverberation from the original signal. Listening experiments show that the method is plausible and the quality of the method is very good when the listener is inside the specific cylinder-shaped surface.
ER  -