Musical Instrument Synthesis and Morphing in Multidimensional Latent Space Using Variational, Convolutional Recurrent Autoencoders

Çakir E, Virtanen T. Musical Instrument Synthesis and Morphing in Multidimensional Latent Space Using Variational, Convolutional Recurrent Autoencoders. In: AES Convention 145. Audio Engineering Society; 2018. Paper 10035. Available from: https://aes.org/publications/elibrary-page/?id=19761

@inproceedings{Cakir2018_19761,
  author = {Çakir, Emre and Virtanen, Tuomas},
  title = {{Musical Instrument Synthesis and Morphing in Multidimensional Latent Space Using Variational, Convolutional Recurrent Autoencoders}},
  booktitle = {AES Convention 145},
  note = {Paper 10035},
  year = {2018},
  month = oct,
  publisher = {Audio Engineering Society},
  url = {https://aes.org/publications/elibrary-page/?id=19761}
}

TY  - CPAPER
TI  - Musical Instrument Synthesis and Morphing in Multidimensional Latent Space Using Variational, Convolutional Recurrent Autoencoders
AU  - Çakir, Emre
AU  - Virtanen, Tuomas
T2  - AES Convention 145
M1  - Paper 10035
PY  - 2018
DA  - 2018/10/06
UR  - https://aes.org/publications/elibrary-page/?id=19761
PB  - Audio Engineering Society
LA  - en
AB  - In this work we propose a deep learning based method—namely, variational, convolutional recurrent autoencoders (VCRAE)—for musical instrument synthesis. This method utilizes the higher level time-frequency representations extracted by the convolutional and recurrent layers to learn a Gaussian distribution in the training stage, which will be later used to infer unique samples through interpolation of multiple instruments in the usage stage. The reconstruction performance of VCRAE is evaluated by proxy through an instrument classifier and provides significantly better accuracy than two other baseline autoencoder methods. The synthesized samples for the combinations of 15 different instruments are available on the companion website.
ER  -