Word based end-to-end real time neural audio effects for equalisation

Balasubramaniam DRK, Timoney J. Word based end-to-end real time neural audio effects for equalisation. In: AES Convention 155. Audio Engineering Society; 2023. Paper 10681. Available from: https://aes.org/publications/elibrary-page/?id=22262

@inproceedings{Balasubramaniam2023_22262,
  author = {Balasubramaniam, Dharanipathi Rathna Kumar and Timoney, Joseph},
  title = {{Word based end-to-end real time neural audio effects for equalisation}},
  booktitle = {AES Convention 155},
  note = {Paper 10681},
  year = {2023},
  month = oct,
  publisher = {Audio Engineering Society},
  url = {https://aes.org/publications/elibrary-page/?id=22262}
}

TY  - CPAPER
TI  - Word based end-to-end real time neural audio effects for equalisation
AU  - Balasubramaniam, Dharanipathi Rathna Kumar
AU  - Timoney, Joseph
T2  - AES Convention 155
M1  - Paper 10681
PY  - 2023
DA  - 2023/10/06
UR  - https://aes.org/publications/elibrary-page/?id=22262
PB  - Audio Engineering Society
LA  - en
AB  - Audio production, typically involving the use of tools such as equalisers and reverberators, can be challenging for non-expert users due to the intricate parameters inherent in these tools’ interfaces. In this paper, we present an end-to-end neural audio effects model based on the temporal convolutional network (TCN) architecture which processes equalisation based on descriptive terms sourced from a crowdsourced vocabulary of word labels for audio effects, enabling users to communicate their audio production objectives with ease. This approach enables users to express their audio production objectives in descriptive language (e.g., "bright," "muddy," "sharp") rather than relying on technical terminology that may not be intuitive to untrained users. We experimented with two word embedding methods to steer the TCN to produce the desired output. Real-time performance is achieved through the use of TCNs with sparse convolutional kernels and rapidly growing dilations. Objective metrics demonstrate the efficacy of the proposed model in applying the appropriately parameterized effects to audio tracks.
ER  -