Neutone SDK: An Open Source Framework for Neural Audio Processing

Mitcheltree C, Teleaga B, Fyfe A, Masuda N, Schäfer M, Bradic A, Tokui N. Neutone SDK: An Open Source Framework for Neural Audio Processing. In: 2025 AES International Conference on Artificial Intelligence and Machine Learning for Audio. Audio Engineering Society; 2025. Paper 32. Available from: https://aes.org/publications/elibrary-page/?id=23021

@inproceedings{Mitcheltree2025_23021,
  author = {Mitcheltree, Christopher and Teleaga, Bogdan and Fyfe, Andrew and Masuda, Naotake and Schäfer, Matthias and Bradic, Alfie and Tokui, Nao},
  title = {{Neutone SDK: An Open Source Framework for Neural Audio Processing}},
  booktitle = {2025 AES International Conference on Artificial Intelligence and Machine Learning for Audio},
  note = {Paper 32},
  year = {2025},
  month = sep,
  publisher = {Audio Engineering Society},
  url = {https://aes.org/publications/elibrary-page/?id=23021}
}

TY  - CPAPER
TI  - Neutone SDK: An Open Source Framework for Neural Audio Processing
AU  - Mitcheltree, Christopher
AU  - Teleaga, Bogdan
AU  - Fyfe, Andrew
AU  - Masuda, Naotake
AU  - Schäfer, Matthias
AU  - Bradic, Alfie
AU  - Tokui, Nao
T2  - 2025 AES International Conference on Artificial Intelligence and Machine Learning for Audio
M1  - Paper 32
PY  - 2025
DA  - 2025/09/02
UR  - https://aes.org/publications/elibrary-page/?id=23021
PB  - Audio Engineering Society
LA  - en
AB  - Neural audio processing has unlocked novel methods of sound transformation and synthesis, yet integrating deep learning models into digital audio workstations (DAWs) remains challenging due to real-time / neural network inference constraints and the complexities of plugin development. In this paper, we introduce the Neutone SDK: an open source framework that streamlines the deployment of PyTorch-based neural audio models for both real-time and offline applications. By encapsulating common challenges such as variable buffer sizes, sample rate conversion, delay compensation, and control parameter handling within a unified, model-agnostic interface, our framework enables seamless interoperability between neural models and host plugins while allowing users to work entirely in Python. We provide a technical overview of the interfaces needed to accomplish this, as well as the corresponding SDK implementations. We also demonstrate the SDK's versatility across applications such as audio effect emulation, timbre transfer, and sample generation, as well as its adoption by researchers, educators, companies, and artists alike. The Neutone SDK is available at github.com/Neutone/neutone_sdk.
ER  -