Low Latency Timbre Interpolation and Warping using Autoencoding Neural Networks

Colonel J, Keene S. Low Latency Timbre Interpolation and Warping using Autoencoding Neural Networks. In: AES Convention 149. Audio Engineering Society; 2020. Paper 10406. Available from: https://aes.org/publications/elibrary-page/?id=20943

@inproceedings{Colonel2020_20943,
  author = {Colonel, Joseph and Keene, Sam},
  title = {{Low Latency Timbre Interpolation and Warping using Autoencoding Neural Networks}},
  booktitle = {AES Convention 149},
  note = {Paper 10406},
  year = {2020},
  month = oct,
  publisher = {Audio Engineering Society},
  url = {https://aes.org/publications/elibrary-page/?id=20943}
}

TY  - CPAPER
TI  - Low Latency Timbre Interpolation and Warping using Autoencoding Neural Networks
AU  - Colonel, Joseph
AU  - Keene, Sam
T2  - AES Convention 149
M1  - Paper 10406
PY  - 2020
DA  - 2020/10/06
UR  - https://aes.org/publications/elibrary-page/?id=20943
PB  - Audio Engineering Society
LA  - en
AB  - A lightweight algorithm for low latency timbre interpolation of two input audio streams using an autoencoding neural network is presented. Short-time Fourier transform magnitude frames of each audio stream are encoded, and a new interpolated representation is created within the autoencoder’s latent space. This new representation is passed to the decoder, which outputs a spectrogram. An initial phase estimation for the new spectrogram is calculated using the original phase of the two audio streams. Inversion to the time domain is done using a Griffin-Lim iteration. A method for avoiding pops between processed batches is discussed. An open source implementation in Python is made available.
ER  -