Time-Varying Audio Effect Modeling by End-to-End Adversarial Training

Bourdin Y, Legrand P, Roche F. Time-Varying Audio Effect Modeling by End-to-End Adversarial Training. J Audio Eng Soc. 2026;74(7/8):555-567. doi:10.17743/jaes.2026.0279

@article{Bourdin2026_23392,
  author = {Bourdin, Yann and Legrand, Pierrick and Roche, Fanny},
  title = {{Time-Varying Audio Effect Modeling by End-to-End Adversarial Training}},
  journal = {Journal of the Audio Engineering Society},
  volume = {74},
  number = {7/8},
  pages = {555--567},
  year = {2026},
  month = jul,
  publisher = {Audio Engineering Society},
  doi = {10.17743/jaes.2026.0279},
  url = {https://doi.org/10.17743/jaes.2026.0279}
}

TY  - JOUR
TI  - Time-Varying Audio Effect Modeling by End-to-End Adversarial Training
AU  - Bourdin, Yann
AU  - Legrand, Pierrick
AU  - Roche, Fanny
T2  - Journal of the Audio Engineering Society
J2  - J. Audio Eng. Soc.
VL  - 74
IS  - 7/8
SP  - 555
EP  - 567
PY  - 2026
DA  - 2026/07/20
DO  - 10.17743/jaes.2026.0279
UR  - https://doi.org/10.17743/jaes.2026.0279
PB  - Audio Engineering Society
LA  - en
AB  - Deep learning has become a standard approach for the modeling of audio effects, yet strictly black-box modeling remains problematic for time-varying systems. Unlike time-invariant effects, training models on devices with internal modulation typically requires the recording or extraction of control signals to ensure the time-alignment required by standard loss functions. This paper introduces a generative adversarial network framework to model such effects using only input-output audio recordings, without requiring a modulation signal extraction. The authors propose a convolutional-recurrent architecture trained via a two-stage strategy: an initial adversarial phase allows the model to learn the distribution of the modulation behavior without strict phase constraints, followed by a supervised fine-tuning phase, where a state prediction network estimates the initial internal states required to synchronize the model with the target. Additionally, a new metric based on chirp-train signals is developed to quantify modulation accuracy. Experiments modeling a vintage hardware phaser demonstrate the method’s ability to capture time-varying dynamics in a fully black-box context.
ER  -