Distilling DDSP: Exploring Real-Time Audio Generation on Embedded Systems

Giudici GA, Caspe F, Gabrielli L, Squartini S, Turchet L. Distilling DDSP: Exploring Real-Time Audio Generation on Embedded Systems. J Audio Eng Soc. 2025;73(6):331-345. doi:10.17743/jaes.2022.0211

@article{Giudici2025_22916,
  author = {Giudici, Gregorio Andrea and Caspe, Franco and Gabrielli, Leonardo and Squartini, Stefano and Turchet, Luca},
  title = {{Distilling DDSP: Exploring Real-Time Audio Generation on Embedded Systems}},
  journal = {Journal of the Audio Engineering Society},
  volume = {73},
  number = {6},
  pages = {331--345},
  year = {2025},
  month = jun,
  publisher = {Audio Engineering Society},
  doi = {10.17743/jaes.2022.0211},
  url = {https://doi.org/10.17743/jaes.2022.0211}
}

TY  - JOUR
TI  - Distilling DDSP: Exploring Real-Time Audio Generation on Embedded Systems
AU  - Giudici, Gregorio Andrea
AU  - Caspe, Franco
AU  - Gabrielli, Leonardo
AU  - Squartini, Stefano
AU  - Turchet, Luca
T2  - Journal of the Audio Engineering Society
J2  - J. Audio Eng. Soc.
VL  - 73
IS  - 6
SP  - 331
EP  - 345
PY  - 2025
DA  - 2025/06/04
DO  - 10.17743/jaes.2022.0211
UR  - https://doi.org/10.17743/jaes.2022.0211
PB  - Audio Engineering Society
LA  - en
AB  - This paper investigates the feasibility of running neural audio generative models on embedded systems, by comparing the performance of various models and evaluating their trade-offs in audio quality, inference speed, and memory usage. This work focuses on differentiable digital signal processing (DDSP) models, due to their hybrid architecture, which combines the efficiency and interoperability of traditional DSP with the flexibility of neural networks. In addition, the application of knowledge distillation (KD) is explored to improve the performance of smaller models. Two types of distillation strategies were implemented and evaluated: audio distillation and control distillation. These methods were applied to three foundation DDSP generative models that integrate Harmonic-Plus-Noise, FM, and Wavetable synthesis. The results demonstrate the overall effectiveness of KD: the authors were able to train student models that are up to 100× smaller than their teacher counterparts while maintaining comparable performance and significantly improving inference speed and memory efficiency. However, cases where KD failed to improve or even degrade student performance have also been observed. The authors provide a critical reflection on the advantages and limitations of KD, exploring its application in diverse use cases and emphasizing the need for carefully tailored strategies to maximize its potential.
ER  -