Neural Proxies for Sound Synthesizers: Learning Perceptually Informed Preset Representations

Combes P, Weinzierl S, Obermayer K. Neural Proxies for Sound Synthesizers: Learning Perceptually Informed Preset Representations. J Audio Eng Soc. 2025;73(9):561-577. doi:10.17743/jaes.2022.0219

@article{Combes2025_22956,
  author = {Combes, Paolo and Weinzierl, Stefan and Obermayer, Klaus},
  title = {{Neural Proxies for Sound Synthesizers: Learning Perceptually Informed Preset Representations}},
  journal = {Journal of the Audio Engineering Society},
  volume = {73},
  number = {9},
  pages = {561--577},
  year = {2025},
  month = sep,
  publisher = {Audio Engineering Society},
  doi = {10.17743/jaes.2022.0219},
  url = {https://doi.org/10.17743/jaes.2022.0219}
}

TY  - JOUR
TI  - Neural Proxies for Sound Synthesizers: Learning Perceptually Informed Preset Representations
AU  - Combes, Paolo
AU  - Weinzierl, Stefan
AU  - Obermayer, Klaus
T2  - Journal of the Audio Engineering Society
J2  - J. Audio Eng. Soc.
VL  - 73
IS  - 9
SP  - 561
EP  - 577
PY  - 2025
DA  - 2025/09/05
DO  - 10.17743/jaes.2022.0219
UR  - https://doi.org/10.17743/jaes.2022.0219
PB  - Audio Engineering Society
LA  - en
AB  - Deep learning appears as an appealing solution for automatic synthesizer programming (ASP), which aims to assist musicians and sound designers in programming sound synthesizers. However, integrating software synthesizers into training pipelines is challenging due to their potential nondifferentiability. This work tackles this challenge by introducing a method to approximate arbitrary synthesizers. Specifically, a neural network is trained to map synthesizer presets onto an audio embedding space derived from a pretrained model. This facilitates the definition of a neural proxy that produces compact yet effective representations, thereby enabling the integration of audio embedding loss into neural-based ASP systems for black-box synthesizers. The authors evaluate the representations derived by various pretrained audio models in the context of neural-based methods for ASP and assess the effectiveness of several neural network architectures, including feedforward, recurrent, and transformer-based models, in defining neural proxies. The proposed method is evaluated using both synthetic and handcrafted presets from three popular software synthesizers and assessed its performance in a synthesizer sound-matching downstream task. Although the benefits of the learned representation are nuanced by resource requirements, encouraging results were obtained for all synthesizers, paving the way for future research into the application of synthesizer proxies for neural-based ASP systems.
ER  -