1D Convolutional Layers to Create Frequency-Based Spectral Features for Audio Networks

Nemer E, Vines G. 1D Convolutional Layers to Create Frequency-Based Spectral Features for Audio Networks. In: AES Convention 153. Audio Engineering Society; 2022. Paper 6. Available from: https://aes.org/publications/elibrary-page/?id=21940

@inproceedings{Nemer2022_21940,
  author = {Nemer, Elias and Vines, Greg},
  title = {{1D Convolutional Layers to Create Frequency-Based Spectral Features for Audio Networks}},
  booktitle = {AES Convention 153},
  note = {Paper 6},
  year = {2022},
  month = oct,
  publisher = {Audio Engineering Society},
  url = {https://aes.org/publications/elibrary-page/?id=21940}
}

TY  - CPAPER
TI  - 1D Convolutional Layers to Create Frequency-Based Spectral Features for Audio Networks
AU  - Nemer, Elias
AU  - Vines, Greg
T2  - AES Convention 153
M1  - Paper 6
PY  - 2022
DA  - 2022/10/06
UR  - https://aes.org/publications/elibrary-page/?id=21940
PB  - Audio Engineering Society
LA  - en
AB  - Time-Frequency transformation and spectral representations of audio signals are commonly used in various machine learning applications. Training networks on frequency features such as the Mel-Spectrogram or Chromagram have been proven more effective and convenient than training on time samples. In practical realizations, these features are created on a different processor and/or pre-computed and stored on disk, requiring additional efforts and making it difficult to experiment with various combinations. In this paper, we provide a PyTorch framework for creating spectral features and time-frequency transformation using the built-in trainable conv1d() layer. This allows computing these on-the-fly as part of a larger network and enabling easier experimentation with various parameters. Our work extends the work in the literature developed for that end: First by adding more of these features; and also by allowing the possibility of either training from initialized kernels or training from random values and converging to the desired solution. The code is written as a template of classes and scripts that users may integrate into their own PyTorch classes for various applications.
ER  -