High-Fidelity Noise Reduction with Differentiable Signal Processing

Steinmetz CJ, Walther T, Reiss JD. High-Fidelity Noise Reduction with Differentiable Signal Processing. In: AES Convention 155. Audio Engineering Society; 2023. Paper 153. Available from: https://aes.org/publications/elibrary-page/?id=22307

@inproceedings{Steinmetz2023_22307,
  author = {Steinmetz, Christian J. and Walther, Thomas and Reiss, Joshua D.},
  title = {{High-Fidelity Noise Reduction with Differentiable Signal Processing}},
  booktitle = {AES Convention 155},
  note = {Paper 153},
  year = {2023},
  month = oct,
  publisher = {Audio Engineering Society},
  url = {https://aes.org/publications/elibrary-page/?id=22307}
}

TY  - CPAPER
TI  - High-Fidelity Noise Reduction with Differentiable Signal Processing
AU  - Steinmetz, Christian J.
AU  - Walther, Thomas
AU  - Reiss, Joshua D.
T2  - AES Convention 155
M1  - Paper 153
PY  - 2023
DA  - 2023/10/06
UR  - https://aes.org/publications/elibrary-page/?id=22307
PB  - Audio Engineering Society
LA  - en
AB  - Noise reduction techniques based on deep learning have demonstrated impressive performance in enhancing the overall quality of recorded speech. While these approaches are highly performant, their application in audio engineering can be limited due to a number of factors. These include operation only on speech without support for music, lack of real-time capability, lack of interpretable control parameters, operation at lower sample rates, and a tendency to introduce artifacts. On the other hand, signal processing-based noise reduction algorithms offer fine-grained control and operation on a broad range of content, however, they often require manual operation to achieve the best results. To address the limitations of both approaches, in this work we introduce a method that leverages a signal processing-based denoiser that when combined with a neural network controller, enables fully automatic and high-fidelity noise reduction on both speech and music signals. We evaluate our proposed method with objective metrics and a perceptual listening test. Our evaluation reveals that speech enhancement models can be extended to music, however training the model to remove only stationary noise is critical. Furthermore, our proposed approach achieves performance on par with the deep learning models, while being significantly more efficient and introducing fewer artifacts in some cases. Listening examples are available online: https://tape.it/research/denoiser
ER  -