Combining Deep Learning and Web Audio for Automated Real-time Audio Speech Production

Sigvardson T. Combining Deep Learning and Web Audio for Automated Real-time Audio Speech Production. In: AES Convention 153. Audio Engineering Society; 2022. Paper 10616. Available from: https://aes.org/publications/elibrary-page/?id=21945

@inproceedings{Sigvardson2022_21945,
  author = {Sigvardson, Tor},
  title = {{Combining Deep Learning and Web Audio for Automated Real-time Audio Speech Production}},
  booktitle = {AES Convention 153},
  note = {Paper 10616},
  year = {2022},
  month = oct,
  publisher = {Audio Engineering Society},
  url = {https://aes.org/publications/elibrary-page/?id=21945}
}

TY  - CPAPER
TI  - Combining Deep Learning and Web Audio for Automated Real-time Audio Speech Production
AU  - Sigvardson, Tor
T2  - AES Convention 153
M1  - Paper 10616
PY  - 2022
DA  - 2022/10/06
UR  - https://aes.org/publications/elibrary-page/?id=21945
PB  - Audio Engineering Society
LA  - en
AB  - With the advent of Web Audio the compliant browser offers a toolbox of audio production components right out of the box. This could prove valuable for any content producer in the audio field, either professional or amateur. In this paper we will add deep learning to this scenario with the ultimate goal of obtaining machine assisted real-time audio production in-browser. As a proof of concept we will implement a basic yet complete one-channel design that uses deep learning to assist an automatic filtering algorithm producing parameters to adjust an audio signal in the Web Audio context of a browser. To achieve this we will evaluate five-class audio prediction models and compare their accuracy at the model building stage with their accuracy when exported to the real-time context. We will present two ways to measure this accuracy in real-time. We will also present a method to reduce jumpiness in real-time predictions when classification scores are ambiguous. We will highlight some important limitations and we will also present a refined model – designed for our domain specific audio set – based on some architectures from previous research and find that that our model outperforms these architectures
ER  -