Modal Representations for Audio Deep Learning

Skare T, Abel JS, Smith IJO. Modal Representations for Audio Deep Learning. In: AES Convention 147. Audio Engineering Society; 2019. Paper 10248. Available from: https://aes.org/publications/elibrary-page/?id=20621

@inproceedings{Skare2019_20621,
  author = {Skare, Travis and Abel, Jonathan S. and Smith, III, Julius O.},
  title = {{Modal Representations for Audio Deep Learning}},
  booktitle = {AES Convention 147},
  note = {Paper 10248},
  year = {2019},
  month = oct,
  publisher = {Audio Engineering Society},
  url = {https://aes.org/publications/elibrary-page/?id=20621}
}

TY  - CPAPER
TI  - Modal Representations for Audio Deep Learning
AU  - Skare, Travis
AU  - Abel, Jonathan S.
AU  - Smith, III, Julius O.
T2  - AES Convention 147
M1  - Paper 10248
PY  - 2019
DA  - 2019/10/06
UR  - https://aes.org/publications/elibrary-page/?id=20621
PB  - Audio Engineering Society
LA  - en
AB  - Deep learning models for both discriminative and generative tasks have a choice of domain representation. For audio, candidates are often raw waveform data, spectral data, transformed spectral data, or perceptual features. For deep learning tasks related to modal synthesizers or processors, we propose new, modal representations for data. We experiment with representations such as an N-hot binary vector of frequencies, or learning a set of modal filterbank coefficients directly. We use these representations discriminatively–classifying cymbal model based on samples–as well as generatively. An intentionally naive application of a basic modal representation to a CVAE designed for MNIST digit images quickly yielded results, which we found surprising given less prior success when using traditional representations like a spectrogram image. We discuss applications for Generative Adversarial Networks, towards creating a modal reverberator generator.
ER  -